Re:Understanding deep learning requires rethinking generaliz...
by magicalhippo on Thursday March 04, 2021 @04:00PM
(#26348249)
Ah hmm yes, good point. I forgot this piece from the article:
We observe a steady deterioration of the generalization error as we increase the noise level. This shows that neural networks are able to capture the remaining signal in the data while at the same time fit the noisy part using brute-force.
Since they score well on the test data, they must have generalized to some degree. But since they're just as good at training on random input they also have the capacity to just memorize the training data.
We observe a steady deterioration of the generalization error as we increase the noise level. This shows that neural networks are able to capture the remaining signal in the data while at the same time fit the noisy part using brute-force.
Since they score well on the test data, they must have generalized to some degree. But since they're just as good at training on random input they also have the capacity to just memorize the training data.