News for hackers. Stuff that matters.
Re:An overview of gradient descent optimization algorithms by highd on Wednesday November 09, 2016 @04:08PM (#12914671)
Blog spam pass through to the original: http://sebastianruder.com/optimizing-gradient-descent/

Useful reference, 6 months old.

Side note, anybody aware of any implementations of the "learning to learn gradient descent by gradient descent" work [0]? I'd love to boost my training time, but I'm worried implementing it myself will just result in an additional set of hyperparameters to tune.

[0] https://arxiv.org/abs/1606.04474

Re:An overview of gradient descent optimization algorithms by zo7 on Wednesday November 09, 2016 @04:17PM (#12914753)
Is it possible to swap the link on this post? That's kind of silly that they just copy the original post. Understanding different optimizers and why they work is important though, especially since often people will just use them without considering what they're actually doing.