Re:An overview of gradient descent optimization algorithms
by zo7 on Wednesday November 09, 2016 @04:17PM
(#12914753)
Is it possible to swap the link on this post? That's kind of silly that they just copy the original post. Understanding different optimizers and why they work is important though, especially since often people will just use them without considering what they're actually doing.
Useful reference, 6 months old.
Side note, anybody aware of any implementations of the "learning to learn gradient descent by gradient descent" work [0]? I'd love to boost my training time, but I'm worried implementing it myself will just result in an additional set of hyperparameters to tune.
[0] https://arxiv.org/abs/1606.04474