Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
by sdpmas on Friday March 20, 2026 @01:08AM
(#47450738)
oh ensemble can be distilled to a single model easily.
[ Parent
| on HN ]
Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
by SknCode on Friday March 20, 2026 @04:17AM
(#47451862)
How?
Re:NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
by sigmoid10 on Friday March 20, 2026 @07:00AM
(#47452925)
Same way you distill any model. Training data efficiency matters only while you train the source model/ensemble. Once you have that you are purely compute bound during distillation.