News for hackers. Stuff that matters.

A comment on OpenAI O3 breakthrough high score on ARC-AGI-PUB. Parent comment

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by whynotminot on Friday December 20, 2024 @09:26PM (#42477046)
“Objective benchmarks are useless, let’s argue about which one works better for me personally.”
[ Parent | on HN ]
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by csomar on Friday December 20, 2024 @10:55PM (#42477440)
Yes. My benchmarks and their benchmarks means AGI. Their benchmarks only means over-fitted.
[ Parent | on HN ]
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by whynotminot on Friday December 20, 2024 @11:02PM (#42477466)
Ok so what if we get different results for our own personal benchmarks/use cases.

(See why objective benchmarks exist?)

[ Parent | on HN ]
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by bakugo on Friday December 20, 2024 @09:54PM (#42477176)
Yes, "objective" benchmarks can be gamed, real-life tasks cannot.
[ Parent | on HN ]