News for hackers. Stuff that matters.

A comment on ChatGPT for Teams. Parent comment

Re:ChatGPT for Teams by epups on Thursday January 11, 2024 @02:36AM (#38948969)
> Keep in mind that modern quantitative approaches to LLM evaluation have been effectively co-designed with the rise of OpenAI, and folks like Ravenwolf routinely disagree with the leaderboards.

Sorry but you're talking complete nonsense here. The benchmark by LMSys (chatbot arena) cannot be gamed, and Ravenwolf is a random-ass poster with no scientific rigor to his benchmarks.

[ Parent | on HN ]
Re:ChatGPT for Teams by ParetoOptimal on Thursday January 11, 2024 @09:17AM (#38952442)
Cannot be gamed? C'mon now... You could pay a bunch of people to vote for your model in the arena.
[ Parent | on HN ]
Re:ChatGPT for Teams by epups on Thursday January 11, 2024 @09:41AM (#38952762)
No you can't, because you actually don't know which model is which when you vote.
[ Parent | on HN ]
Re:ChatGPT for Teams by ParetoOptimal on Thursday January 11, 2024 @11:27AM (#38954413)
Do only the initial votes count? Because after I made an initial choice I was then put in a session where I saw the name of both of the AI. I made subsequent votes in that session where I could see their names.
[ Parent | on HN ]
Re:ChatGPT for Teams by epups on Thursday January 11, 2024 @01:57PM (#38956970)
https://github.com/lm-sys/FastChat/issues/1210
[ Parent | on HN ]