Re:OpenAI dropped the price of o3 by 80%
by esafak on Tuesday June 10, 2025 @02:09PM
(#44239622)
Are there any benchmarks that track historical performance?
[ Parent
| on HN ]
good question, and I don't know of any, although it's a no brainer that someone should make it.
a proxy to that may be the anecdotal evidence of users who report back in a month that model X has gotten dumber (started with gpt-4 and keeps happening, esp. with Anthro and OpenAI models). I haven't heard such anecdotal stories about Gemini, R1, etc.
Re:OpenAI dropped the price of o3 by 80%
by SparkyMcUnicorn on Tuesday June 10, 2025 @03:17PM
(#44240357)
Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.
Re:OpenAI dropped the price of o3 by 80%
by __mharrison__ on Tuesday June 10, 2025 @04:02PM
(#44240843)
Updated yesterday... https://aider.chat/docs/leaderboards/
That Deepseek price is always hilarious to see in these charts.
Re:OpenAI dropped the price of o3 by 80%
by SparkyMcUnicorn on Tuesday June 10, 2025 @06:26PM
(#44242178)
That's not the one I'm referring to. See my other comments or your sibling comment.