Re:NanoGPT Slowrun: Language Modeling with Limited Data, Inf...
by timshel1 on Wednesday March 04, 2026 @05:09PM
(#47254652)
Modded-nanogpt is also much more data efficient than vanilla napogpt, even if some of the individual optimizations trade off higher throughput for worse data efficiency.
[ Parent
| on HN ]
Re:NanoGPT Slowrun: Language Modeling with Limited Data, Inf...
by sdpmas on Wednesday March 04, 2026 @05:32PM
(#47254918)
yes, agreed, modded-nanogpt is already a data-efficient variant of original nanogpt. just that the kinds of algorithms it allows are somewhat constrained because it optimizes for wall clock time.