News for hackers. Stuff that matters.
Re:Why deep learning works even though it shouldn’t by machinelearning on Wednesday October 21, 2020 @06:12PM (#24852807)
I think you misunderstood the point of deep double descent. The x-axis is not number of training epochs, it is model capacity.

I think you'd be interested in https://arxiv.org/abs/1611.03530. It discusses how SGD is an implicit regularizer. We also actually want high variance weights for symmetry breaking.

There are no comments on this comment yet.