News for hackers. Stuff that matters.
Re:Every Model Learned by Gradient Descent Is Approximately ... by kaczordon on Sunday December 06, 2020 @12:20PM (#25324913)
Doesn’t gradient descent use a convex cost function so that it always generates a global minimum?
Re:Every Model Learned by Gradient Descent Is Approximately ... by wxnx on Sunday December 06, 2020 @12:32PM (#25325002)
No, not necessarily. The objective functions used to train neural networks are generally non-convex (the nets themselves being non-convex as well), but are traditionally trained using stochastic gradient descent (and its variants).
Re:Every Model Learned by Gradient Descent Is Approximately ... by mrtranscendence on Sunday December 06, 2020 @01:58PM (#25325670)
This really threw me off when I first started learning about ANNs, coming from a traditional econometrics background. B-but ... the parameters aren’t identified!