Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
by SknCode on Friday March 20, 2026 @04:17AM
(#47451862)
How?
[ Parent
| on HN ]
Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
by sigmoid10 on Friday March 20, 2026 @07:00AM
(#47452925)
Same way you distill any model. Training data efficiency matters only while you train the source model/ensemble. Once you have that you are purely compute bound during distillation.