Re:NanoGPT Slowrun: Language Modeling with Limited Data, Inf...
by alyxya on Wednesday March 04, 2026 @08:21PM
(#47256258)
Fundamentally I don't believe second-order methods get better data efficiency by itself, but changes to the optimizer can because the convergence behavior changes. ML theory lags behind the results in practice.
[ Parent
| on HN ]
There are no comments on this comment yet.