Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by jerpint on Tuesday May 28, 2024 @04:01PM
(#40504940)
Completely agree, and itβs been great to see to some extent that the gap between open source and 3rd party models is really starting to close
Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by FergusArgyll on Tuesday May 28, 2024 @05:25PM
(#40505779)
Someone made a framework to make and test llms on your own benchmarks
you might find it useful
https://github.com/carlini/yet-another-applied-llm-benchmark
The other one I consult occasionally is https://gorilla.cs.berkeley.edu/leaderboard.html - the Berkeley Function-Calling Leaderboard - because function calling is a really useful characteristic of smaller models.
I believe the best ways to evaluate these models is with your own set of task-specific evals for your own project, but these are REALLY difficult to construct. I still don't have a good solution there for my own work.