Re:Gemini 2.0: our new AI model for the agentic era
by gbickford on Wednesday December 11, 2024 @02:59PM
(#42392153)
Small models don't "know" as much so they hallucinate more. They are better suited for generations that are based in a ground truth, like in a RAG setup.
A better comparison might be Flash 2.0 vs 4o-mini. Even then, the models aren't meant to have vast world knowledge, so benchmarking them on that isn't a great indicator of how they would be used in real-world cases.
Re:Gemini 2.0: our new AI model for the agentic era
by ipsum2 on Wednesday December 11, 2024 @04:21PM
(#42393114)
Yes, it's not an apples to apples comparison. My point is the position it's at on the lmarena leaderboard is misplaced due to the hallucination issues.
A better comparison might be Flash 2.0 vs 4o-mini. Even then, the models aren't meant to have vast world knowledge, so benchmarking them on that isn't a great indicator of how they would be used in real-world cases.