News for hackers. Stuff that matters.

A comment on Gemini 2.0: our new AI model for the agentic era.

Re:Gemini 2.0: our new AI model for the agentic era by ipsum2 on Wednesday December 11, 2024 @01:03PM (#42390719)
Tested out Gemini-2 Flash, I had such high hopes that a better base model would help. It still hallucinates like crazy compared to GPT-4o.
[ Parent | on HN ]
Re:Gemini 2.0: our new AI model for the agentic era by gbickford on Wednesday December 11, 2024 @02:59PM (#42392153)
Small models don't "know" as much so they hallucinate more. They are better suited for generations that are based in a ground truth, like in a RAG setup.

A better comparison might be Flash 2.0 vs 4o-mini. Even then, the models aren't meant to have vast world knowledge, so benchmarking them on that isn't a great indicator of how they would be used in real-world cases.

[ Parent | on HN ]
Re:Gemini 2.0: our new AI model for the agentic era by ipsum2 on Wednesday December 11, 2024 @04:21PM (#42393114)
Yes, it's not an apples to apples comparison. My point is the position it's at on the lmarena leaderboard is misplaced due to the hallucination issues.
[ Parent | on HN ]