News for hackers. Stuff that matters.
Re:Myths in Machine Learning Research by kalenx on Monday February 25, 2019 @09:03PM (#19251406)
It's a deadly sin considering that many (most?) researchers do not treat it as a test set, but as a "validation set #2". Basically, you tune your hyperparameters (up to the random seed!) to fare better on the test set. So, as shown in the cited paper, the results are not generalization results anymore.

There are no comments on this comment yet.