News for hackers. Stuff that matters.
Re:NanoGPT Slowrun: Language Modeling with Limited Data, Inf... by jbergqvist on Thursday March 05, 2026 @03:48AM (#47259244)
Very interesting benchmark, excited to see what comes out of this. Considering humans are enourmously more sample efficient compared to today's models, it seems clear there's a lot of room to close that gap. The fact that they hit 5.5x in the first week with relatively straightforward changes suggests we're nowhere near the ceiling for data efficiency
Re:NanoGPT Slowrun: Language Modeling with Limited Data, Inf... by sdpmas on Thursday March 05, 2026 @03:50AM (#47259260)
absolutely!