News for hackers. Stuff that matters.
Re:Karpathy 2025 LLM Year in Review by MrOrelliOReilly on Saturday December 20, 2025 @09:18AM (#46336373)
Yes, I personally feel that the "official" benchmarks are increasingly diverging from the everyday reality of using these models. My theory is that we are reaching a point where all the models are intelligent enough for day-to-day queries, so points like style/personality and proper use of web queries and other capabilities are better differentiators than intelligence alone.
Re:Karpathy 2025 LLM Year in Review by int_19h on Sunday December 21, 2025 @06:36PM (#46349772)
The benchmarks haven't reflected the real utility for a very long time. At best they tell you which models are definitely bad.