News for hackers. Stuff that matters.
Re:Karpathy 2025 LLM Year in Review by ramoz on Saturday December 20, 2025 @09:06AM (#46336288)
https://x.com/giansegato/status/2002203155262812529/photo/1

https://x.com/METR_Evals/status/2002203627377574113

> Even Google's cheaper Gemini 3 Flash model seems to be slightly ahead of Opus 4.5.

What an insane take for anybody uses these models daily.

Re:Karpathy 2025 LLM Year in Review by MrOrelliOReilly on Saturday December 20, 2025 @09:18AM (#46336373)
Yes, I personally feel that the "official" benchmarks are increasingly diverging from the everyday reality of using these models. My theory is that we are reaching a point where all the models are intelligent enough for day-to-day queries, so points like style/personality and proper use of web queries and other capabilities are better differentiators than intelligence alone.
Re:Karpathy 2025 LLM Year in Review by int_19h on Sunday December 21, 2025 @06:36PM (#46349772)
The benchmarks haven't reflected the real utility for a very long time. At best they tell you which models are definitely bad.