Understanding deep learning requires rethinking generalization

Re:Understanding deep learning requires rethinking generaliz... by joe_the_user on Thursday March 04, 2021 @10:36PM (#26352287)

The paper says that: "Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data"

What is a generalization of a random labeling? Does the network become psychic? Is this saying "garbage works fine by these measures"?

[ Parent | on HN ]

Re:Understanding deep learning requires rethinking generaliz... by visarga on Friday March 05, 2021 @03:36AM (#26354401)

The network learns how to map data into an intermediate representation, then learns a hash output for each of these intermediate values. So it learns two things compared to when the intermediate values correspond to the task.

[ Parent | on HN ]

Re:Understanding deep learning requires rethinking generaliz... by DavidSJ on Friday March 05, 2021 @10:50AM (#26358159)

Random labeling can’t generalize. But in the usual case, the labels are nonrandom and the learned map generalizes.

[ Parent | on HN ]