Re:A New Lens on Understanding Generalization in Deep Learning
by choppaface on Sunday March 21, 2021 @01:02AM
(#26528240)
or even just an ablation on Imagenet or some other “large” dataset. Did this paper get accepted because it has five pages of citations and invokes Vapnik in the second sentence?
[ Parent
| on HN ]
Re:A New Lens on Understanding Generalization in Deep Learning
by preetum on Sunday March 21, 2021 @10:37AM
(#26530769)
We have ImageNet experiments in Section 4 of the full paper: https://arxiv.org/abs/2010.08127
Re:A New Lens on Understanding Generalization in Deep Learning
by choppaface on Sunday March 21, 2021 @03:04PM
(#26532945)
You're not ablating anything there. What happens when Train Infinity (Train 150K) doubles in size (to Train Infinity_2 -> 300K)? What happens when you add an unseen class? These are real-world conditions that hamper existing theoretical estimation of the generalization gap-- the "ideal world" always gets larger. In Bengio's group paper (Predicting the Generalization Gap https://arxiv.org/pdf/1810.00113.pdf ) they actually do these sorts of ablations.
Also, you use K (thousands) and $K$ (latex K) interchangeably; it's really hard to decipher is K is a variable or what you mean.
Re:A New Lens on Understanding Generalization in Deep Learning
by 6gvONxR4sf7o on Sunday March 21, 2021 @11:40AM
(#26531228)
Why would you criticize them so hard for not doing imagenet experiments when they did do them?
Re:A New Lens on Understanding Generalization in Deep Learning
by choppaface on Sunday March 21, 2021 @03:09PM
(#26532984)
This article was promoted by Google's PR blog and the conference is ultra selective (probably excessively selective). Hyped results deserve extra criticism. And no, they didn't do any useful ablation experiments.