Re:Gradient Descent Finds Global Minima of Deep Neural Networks
by jey on Monday November 12, 2018 @03:48PM
(#18435617)
I interpret it as saying: "non-convexity ain't no big deal in this context", and this is the latest contribution in a line of other "no spurious local minima" machine learning theory research. It's interesting for showing that it applies to a rather complex model like ResNet.
[ Parent
| on HN ]
Re:Gradient Descent Finds Global Minima of Deep Neural Networks
by cosmic_ape on Monday November 12, 2018 @04:08PM
(#18435835)
Its quite important that its not so much about "non-convexity", its more about "random initialization". All these results, and there are others, more sophisticated, basically say that after random init everything is already good, and the gradient descent just has to train the highest layer, without screwing up the previous ones. They are all pretty naive that way.