News for hackers. Stuff that matters.
Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute by SknCode on Friday March 20, 2026 @04:17AM (#47451862)
How?
Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute by sigmoid10 on Friday March 20, 2026 @07:00AM (#47452925)
Same way you distill any model. Training data efficiency matters only while you train the source model/ensemble. Once you have that you are purely compute bound during distillation.