News for hackers. Stuff that matters.
Re:OpenAI O3-Mini by gundmc on Friday January 31, 2025 @02:16PM (#42890736)
If this is the hierarchy, why does 4o score so much higher than o1 on LLM Arena?

Worrisome for OpenAI that Gemini's mini/flash reasoning model outscores both o1 and 4o handily.

Re:OpenAI O3-Mini by crazysim on Friday January 31, 2025 @02:18PM (#42890754)
Is it possible people are voting for speed of responsiveness too?
Re:OpenAI O3-Mini by kgeist on Friday January 31, 2025 @04:19PM (#42892179)
I suspect people on LLM Arena don't ask complex questions too often, and reasoning models seem to perform worse than simple models when the goal is just casual conversation or retrieving embedded knowledge. Reasoning models probably 'overthink' in such cases. And slower, too.
Re:OpenAI O3-Mini by imtringued on Saturday February 01, 2025 @05:46AM (#42897434)
The LLM Arena deletes your prompt when you restart so what's the point in trying to write a complicated prompt and testing an exhaustive number of pairs?

It's easy to pin this on the users, but that website is hostile to putting in any effort.

This is something I've noticed a lot actually. A lot of AI projects just give you an input field and call it a day. Expecting the user to do the heavy lifting.

Re:OpenAI O3-Mini by energy123 on Friday January 31, 2025 @06:04PM (#42893437)
o1 on LLM Arena often times out (network error) while thinking. But they still allow you to vote and they make it seem as if your vote is registered.