Re:Show HN: ARC Prize – a $1M+ competition towards open AGI ...
by slicerdicer1 on Wednesday June 12, 2024 @04:56AM
(#40655996)
AGI is not when the AI is good at some particular thing, AGI is when we have nothing left at which the AI is bad at (compared to humans).
Re:Show HN: ARC Prize – a $1M+ competition towards open AGI ...
by n2d4 on Wednesday June 12, 2024 @06:35AM
(#40656538)
The tasks are handmade. There is no "problem generator".
Re:Show HN: ARC Prize – a $1M+ competition towards open AGI ...
by elicksaur on Wednesday June 12, 2024 @12:04PM
(#40659642)
Yes to your last question, that is essentially how the first iteration solutions operated. Some of the original kaggle competition’s best solutions used a DSL made of these transformations. That was 4 years ago. [1]
The issue with that path is that the problems aren’t using a programmatic generator. The rule sets are anything a person could come up with. It might be as simple as “biggest object turns blue” but they can be much more complicated.
Additionally, the test set is private so it can’t be trained on or extracted from. It has rules that aren’t in the public sets.
[1] https://www.kaggle.com/competitions/abstraction-and-reasonin...
That's a stretch. This is a problem at which LLMs are bad. That does not imply it's a good measure of artificial general intelligence.
After working a few of the problems, I was wondering how many different transformation rules the problem generator has. Not very many, it seems. So the problem breaks down into extracting the set of transformation rules from the data, then applying them to new problems. The first part of that is hard. It's a feature extraction problem. The transformations seem to be applied rigidly, so once you have the transformation rules, and have selected the ones that work for all the input cases, application should be straightforward.
This seems to need explicit feature extraction, rather than the combined feature extraction and exploitation LLMs use. Has anyone extracted the rule set from the test cases yet?