Re:An overview of the theory of overparameterized machine le...
by quocanh on Friday September 24, 2021 @07:05PM
(#28648775)
Ok so according to your theory, we could make this hypothesis: if we applied a neural network to an unseen example (for example, a validation dataset), then we would get accuracy that is equivalent to randomly picking a random label. Well surprise, surprise - we obviously don't get that. So there is clearly more going on than "neural hashing".
You're not answering this problem with unseen data so it's really hard for me to follow your reasoning here.
My point up above is that classification problems are too weak, exactly because these kinds of shortcuts are readily available. The leading edge of ML research is over-focused on ImageNet classification in particular.