Re:Mathematics of Deep Learning [pdf]
by aoeusnth1 on Sunday December 17, 2017 @04:04AM
(#15944922)
Noob here - what do you mean by distributional shift?
[ Parent
| on HN ]
In short, it's when your training data come from different conditions than your real world data.
This means that the probability distribution your model was trained on is different from the one you are using it on. This means that an model that did great in cross-validation could still be useless in practice.
Basically, it's when your distribution that your classes come from change between training and test.
For example, if after training your classifier to distinguish between cats and dogs, all of a sudden society decides that all white haired dogs are now called cats, your classifier is clearly going to get "poor" performance.