News for hackers. Stuff that matters.
Re:Karpathy 2025 LLM Year in Review by dr_dshiv on Saturday December 20, 2025 @08:36AM (#46336085)
https://lmarena.ai/leaderboard/webdev

LM Arena shows Claude Opus 4.5 on top

Re:Karpathy 2025 LLM Year in Review by HarHarVeryFunny on Saturday December 20, 2025 @08:57AM (#46336236)
I wonder how model competence and/or user preference on web development (that leaderboard) carries over to more complex and larger projects, or more generally anything other than web development ?

In addition to whatever they are exposed to as part of pre-training, it'd be interesting to know what kind of coding tasks these models are being RL-trained for? Are things like web development and maybe Python/ML coding overemphasized, or are they also being trained on things like Linux/Windows/embedded development etc in different languages?