News for hackers. Stuff that matters.
Re:What we learned copying all the best code assistants by nathanasmith on Saturday January 04, 2025 @01:45PM (#42596735)
For the lmarena leaderboard to be really useful you need click the "Style Control" button so that it normalizes for LLMs that generate longer answers, etc. that, while humans may find them more stylistically pleasing, and upvote them, the answers often end up being worse. When you do that, o1 comes out on top followed by o1-preview, then Sonnet 3.5, and in fourth place Gemini Preview 1206.

There are no comments on this comment yet.