News for hackers. Stuff that matters.
Re:Mathematics of Deep Learning [pdf] by make3 on Saturday December 16, 2017 @10:37PM (#15943932)
my understanding is that the issue is that the full Hessian of the loss is too expensive to compute at each step for the relative size of the increase in learning speed
Re:Mathematics of Deep Learning [pdf] by _0ffh on Sunday December 17, 2017 @06:04AM (#15945251)
Yeah I think that's why quasi-Newton methods like BFGS have been developed.