Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB
by versteegen on Friday December 20, 2024 @07:12PM
(#42476408)
ARC wasn't designed as a benchmark for LLMs, and it doesn't make much sense to compare them on it since it's the wrong modality. Even a MLM with image inputs can't be expected to do well, since they're nothing like 99.999% of the training data. The fact that even a text-only LLM can solve ARC problems with the proper framework is important, however.
[ Parent
| on HN ]
There are no comments on this comment yet.