Gradient Descent Finds Global Minima of Deep Neural Networks

Re:Gradient Descent Finds Global Minima of Deep Neural Networks by cosmic_ape on Monday November 12, 2018 @03:48PM (#18435616)

This is not necessarily a trivial fact, but I wouldn't call it incredible. It says a net trained with gradient descent can fit the input perfectly if its much larger than the input.

But this says nothing about generalization, i.e. performance on test set -- which is what we really want.

[ Parent | on HN ]

Re:Gradient Descent Finds Global Minima of Deep Neural Networks by iandanforth on Monday November 12, 2018 @03:54PM (#18435686)

Importantly not SGD, just GD.

[ Parent | on HN ]

Re:Gradient Descent Finds Global Minima of Deep Neural Networks by cosmic_ape on Monday November 12, 2018 @04:02PM (#18435787)

you're right, fixed.

[ Parent | on HN ]