Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB
by modeless on Friday December 20, 2024 @04:10PM
(#42475068)
You're right, I was wrong to say "most challenging". I think the correct statement would be "most challenging long-standing benchmark" as I don't believe any other test designed in 2019 has resisted progress for so long.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB
by esafak on Saturday December 21, 2024 @09:07AM
(#42479758)
They should put some famous, unsolved problems in the next edition so ML researchers do some actually useful work while they're "gaming" the benchmarks :)
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB
by modeless on Monday December 23, 2024 @04:41PM
(#42497833)
I'm certain that the big labs will be gunning for the Millenium Prize problems.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB
by pynappo on Friday December 20, 2024 @03:44PM
(#42474838)
Apparently o3 scored about 25%
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB
by FiberBundle on Friday December 20, 2024 @04:06PM
(#42475025)
This is actually the result that I find way more impressive. Elite mathematicians think these problems are challenging and thought they were years away from being solvable by AI.
This[1] is currently the most challenging benchmark. I would like to see how O3 handles it, as O1 solved only 1%.
1. https://epoch.ai/frontiermath/the-benchmark