News for hackers. Stuff that matters.
Re:OpenAI dropped the price of o3 by 80% by Bjorkbat on Wednesday June 11, 2025 @08:06AM (#44246736)
Point remains though, they crushed the benchmark using a specialized model that you’ll probably never have access to, whether personally or through a company.

They inflated expectations and then released to the public a model that underperforms

Re:OpenAI dropped the price of o3 by 80% by throwaway314155 on Wednesday June 11, 2025 @09:42AM (#44247543)
They revealed the price points for running those evaluations. IIRC the "high" level of reasoning cost tens of thousands of dollars if not more. I don't think they really inflated expectations. In fact a lot of what we learned is that ARC-AGI probably isn't a very good AGI evaluation (it claims to not be one, but the name suggests otherwise).