News for hackers. Stuff that matters.
Re:NanoGPT Slowrun: Language Modeling with Limited Data, Inf... by alyxya on Wednesday March 04, 2026 @08:21PM (#47256258)
Fundamentally I don't believe second-order methods get better data efficiency by itself, but changes to the optimizer can because the convergence behavior changes. ML theory lags behind the results in practice.

There are no comments on this comment yet.