News for hackers. Stuff that matters.
Re:2025: The Year in LLMs by HPMOR on Thursday January 01, 2026 @12:11AM (#46451489)
I think this is an open question still and very interesting. Ilya discussed this on the Dwarkesh podcast. But the capabilities of LLMs is clearly exponential and perhaps super exponential. We went from something that could string together incoherent text in 2022 to general models helping people like Terrance Tao and Scott Aaronson write new research papers. LLMs also beat IMO and the ICPC. We have entered the John Henry era for intellectual tasks...
Re:2025: The Year in LLMs by llmslave2 on Thursday January 01, 2026 @12:30AM (#46451577)
> But the capabilities of LLMs is clearly exponential and perhaps super exponential

By what metric?

Re:2025: The Year in LLMs by utopiah on Thursday January 01, 2026 @03:22AM (#46452273)
BS metric... /s
Re:2025: The Year in LLMs by jennyholzer3 on Friday January 02, 2026 @09:20AM (#46464980)
Chat GPT told him it was true
Re:2025: The Year in LLMs by aspenmartin on Friday January 02, 2026 @04:08PM (#46469358)
- Scaling laws (Chinchilla type)

- METR task horizon

It's a mix, performance gains are bursty but we have been getting a lot of bursts (RLVR, test-time compute, agentic breakthroughs)

Re:2025: The Year in LLMs by tsimionescu on Thursday January 01, 2026 @04:25AM (#46452611)
> LLMs also beat IMO and the ICPC

Very spurious claims, given that there was no effort made to check whether the IMO or ICPC problems were in the training set or not, or to quantify how far problems in the training set were from the contest problems. IMO problems are supposed to be unique, but since it's not at the frontier of math research, there is no guarantee that the same problem, or something very similar, was not solved in some obscure manual.