News for hackers. Stuff that matters.
Re:Gemini 2.5 Flash by mupuff1234 on Thursday April 17, 2025 @03:56PM (#43721464)
Not sure "decimated" is a fitting word for "slightly higher performance on some benchmarks".
Re:Gemini 2.5 Flash by fwip on Thursday April 17, 2025 @04:40PM (#43721992)
Perhaps they were using the original meaning of "one-tenth destroyed." :P
Re:Gemini 2.5 Flash by ein0p on Thursday April 17, 2025 @05:17PM (#43722303)
66.8% error rate reduction for o4-mini on AIME2025, and 21% error rate reduction on MMMU isn't "slightly higher". It'll be quite noticeable in practice.