News for hackers. Stuff that matters.
Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute by sdpmas on Thursday March 19, 2026 @03:45PM (#47444836)
> you can simply generate more, and higher quality, artificial data

this is simply not true. and it's very clear if you look at continual learning, robotics, biology, etc. each has enough economic incentives to spend 1000x compute if that led to much better results, but we just don't know how to do that.

good point on chinchilla, but our models are still absurdly large no matter what standards you compare them to.

Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute by littlestymaar on Thursday March 19, 2026 @03:51PM (#47444920)
> this is simply not true. and it's very clear if you look at continual learning, robotics, biology, etc. each has enough economic incentives to spend 1000x compute if that led to much better results, but we just don't know how to do that

I'm (and so is the post itself) talking about LLMs in particular, and this is indeed true for LLM.

Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute by sdpmas on Thursday March 19, 2026 @04:03PM (#47445089)
continual learning is LLMs :) ultimately everything will be/already is data bottlenecked.