News for hackers. Stuff that matters.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by ChildOfChaos on Friday December 20, 2024 @04:53PM (#42475436)
This is insanely expensive to run though. Looks like it cost around $1 million of compute to get that result.

Doesn't seem like such a massive breakthrough when they are throwing so much compute at it, particularly as this is test time compute, it just isn't practical at all, you are not getting this level with a ChatGPT subscription, even the new $200 a month option.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by evouga on Friday December 20, 2024 @07:14PM (#42476415)
Sure but... this is the technology at the most expensive it will ever be. I'm impressed that o3 was able to achieve such high performance at all, and am not too pessimistic about costs decreasing over time.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by MVissers on Friday December 20, 2024 @10:30PM (#42477324)
We've seen 10-100x cost decrease per year since GPT-3 came out for the same capabilities.

So... Next year this tech will most likely be quite a bit cheaper.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by ChildOfChaos on Saturday December 21, 2024 @07:44AM (#42479353)
Even at 100x cost decrease this will still cost $10,000 to beat a benchmark. It won't scale when you have that amount of compute requirements and power.

GPT-3 may massively reduced in cost, but it's requirements were not anyway extreme compared to this.