Re:Every Model Learned by Gradient Descent Is Approximately ...
by kaczordon on Sunday December 06, 2020 @12:20PM
(#25324913)
Doesn’t gradient descent use a convex cost function so that it always generates a global minimum?
[ Parent
| on HN ]
Re:Every Model Learned by Gradient Descent Is Approximately ...
by wxnx on Sunday December 06, 2020 @12:32PM
(#25325002)
No, not necessarily. The objective functions used to train neural networks are generally non-convex (the nets themselves being non-convex as well), but are traditionally trained using stochastic gradient descent (and its variants).
Re:Every Model Learned by Gradient Descent Is Approximately ...
by mrtranscendence on Sunday December 06, 2020 @01:58PM
(#25325670)
This really threw me off when I first started learning about ANNs, coming from a traditional econometrics background. B-but ... the parameters aren’t identified!