Re:Getting 50% (SoTA) on Arc-AGI with GPT-4o
by logicchains on Tuesday June 18, 2024 @02:23AM
(#40714614)
>Yeah humans can generalize much faster than LLM with far fewer "examples" running on sandwiches and coffee.
This isn't really true. If you give an LLM a large prompt detailing a new spoken language, programming language or logical framework with a couple examples, and ask it to do something with it, it'll probably do a lot better at it than if you just let an average human read the same prompt and do the same task.
Re:Getting 50% (SoTA) on Arc-AGI with GPT-4o
by infgeoax on Tuesday June 18, 2024 @08:03PM
(#40723361)
Hmm, but is it really "generalizing" or just pulling information from the training data? I think that's what this benchmark is really about: to adapt to something it has never seen before quickly.
This isn't really true. If you give an LLM a large prompt detailing a new spoken language, programming language or logical framework with a couple examples, and ask it to do something with it, it'll probably do a lot better at it than if you just let an average human read the same prompt and do the same task.