News for hackers. Stuff that matters.
Re:A Theory of Deep Learning by yorwba on Thursday May 07, 2026 @03:12AM (#48046382)
The handwaving required is just to assume a diagonal preconditioner, and the optimal preconditioner under that constraint corresponds to the new update rule. (See section F of the paper.) And of course a diagonal preconditioner works on the per-paramer level.

There are no comments on this comment yet.