Re:Myths in Machine Learning Research
by kalenx on Monday February 25, 2019 @09:03PM
(#19251406)
It's a deadly sin considering that many (most?) researchers do not treat it as a test set, but as a "validation set #2". Basically, you tune your hyperparameters (up to the random seed!) to fare better on the test set. So, as shown in the cited paper, the results are not generalization results anymore.
[ Parent
| on HN ]
There are no comments on this comment yet.