News for hackers. Stuff that matters.

A comment on Gemini 2.5. Parent comment

Re:Gemini 2.5 by flir on Tuesday March 25, 2025 @01:35PM (#43473824)
Man, I hope those benchmarks actually measure something.
[ Parent | on HN ]
Re:Gemini 2.5 by Legend2440 on Tuesday March 25, 2025 @01:40PM (#43473901)
I would say they are a fairly good measure of how well the model has integrated information from pretraining.

They are not so good at measuring reasoning, out-of-domain performance, or creativity.

[ Parent | on HN ]