News for hackers. Stuff that matters.
Re:OpenAI dropped the price of o3 by 80% by esafak on Tuesday June 10, 2025 @02:09PM (#44239622)
Are there any benchmarks that track historical performance?
Re:OpenAI dropped the price of o3 by 80% by behnamoh on Tuesday June 10, 2025 @02:10PM (#44239639)
good question, and I don't know of any, although it's a no brainer that someone should make it.

a proxy to that may be the anecdotal evidence of users who report back in a month that model X has gotten dumber (started with gpt-4 and keeps happening, esp. with Anthro and OpenAI models). I haven't heard such anecdotal stories about Gemini, R1, etc.

Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @03:17PM (#44240357)
Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.
Re:OpenAI dropped the price of o3 by 80% by __mharrison__ on Tuesday June 10, 2025 @04:02PM (#44240843)
Re:OpenAI dropped the price of o3 by 80% by vitaflo on Tuesday June 10, 2025 @04:26PM (#44241098)
That Deepseek price is always hilarious to see in these charts.
Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @06:26PM (#44242178)
That's not the one I'm referring to. See my other comments or your sibling comment.
Re:OpenAI dropped the price of o3 by 80% by esafak on Tuesday June 10, 2025 @03:43PM (#44240642)