Re:An overview of the theory of overparameterized machine le...
by thisiszilff on Tuesday September 21, 2021 @03:24PM
(#28608806)
Yes, you're right. You should never see the test/evaluation dataset during training so it would be impossible to "memorize" the test cases. You would get good near perfect accuracy on the training data, but not the test set. I think the closest analogue would be models that produce conceptual embeddings somewhere in them -- those are kind of like hashes with the property that similar things have similar embeddings. Many classification neural networks kind of operate like that -- the initial layers produce a representation of the data and then the final layer actually performs the classification.
[ Parent
| on HN ]
There are no comments on this comment yet.