Re:A New Lens on Understanding Generalization in Deep Learning
by gwern on Monday March 22, 2021 @10:12AM
(#26541023)
I wonder if that's an important difference? Sales data seems far more noisy and less informative than all of the scaling research datasets I can think of like images or text. It could also just be that you had some minor flaw causing underfitting in your approach like using too-small models which aren't compute-optimal or too much regularization (a critical part of scaling papers is removing standard regularizers like dropout to avoid hobbling the model).
[ Parent
| on HN ]
Re:A New Lens on Understanding Generalization in Deep Learning
by chroem- on Tuesday March 23, 2021 @02:46AM
(#26551824)
I've found that it's surprisingly insensitive to the amount of regularization applied, however it definitely does warrant further experimentation. Also, I expect that one other distinction vs. image data is much higher correlation between samples in the dataset.