Why deep learning works even though it shouldn’t

Re:Why deep learning works even though it shouldn’t by muppet_frog on Tuesday October 20, 2020 @05:26PM (#24842493)

...hmm, that was counter to my understanding (limited though it may be...) which was partially formed by this paper: https://arxiv.org/abs/1712.09913

TLDR - loss landscapes are nasty, but you can tame them with skip connections.

[ Parent | on HN ]

Re:Why deep learning works even though it shouldn’t by blackbear_ on Wednesday October 21, 2020 @02:32AM (#24845272)

These two papers are not necessarily contradicting each other, but perhaps my description was a bit sloppy.

Sagun et al. (and derivative works) only focus on the Hessian on the trajectory followed by gradient descent, while Li et al. give a broader look at the loss surface as a whole.

[ Parent | on HN ]