News for hackers. Stuff that matters.
Re:Overfitting, Regularization, and Hyperparameter Optimization by dswalter on Tuesday December 01, 2015 @01:20PM (#10657566)
OP here: I probably should have made it clearer that the using the same train-test splits is verboten.

Your second and third paragraphs are also exactly correct. I attempted to make those points in the post, but you've done so more effectively here.

Re:Overfitting, Regularization, and Hyperparameter Optimization by dasboth on Wednesday December 02, 2015 @03:26PM (#10665556)
Yeah I just thought it was worth hammering it home, especially because of some literature's use of "test set" to mean "validation set" which can really throw off a beginner like myself.