Re:A popular self-driving car dataset is missing labels for ...
by sukilot on Tuesday February 11, 2020 @05:19PM
(#22303574)
> deep nets will still find the structure in the data
Yes, but in the training data, not in the non-training data! We want the cars to avoid real pedestrians, not only the ones labelled in the training set!
From the paper:
> When trained on a completely random labeling of the true data, neural networks achieve 0 training error. The test error, of course, is no better than random chance as there is no correlation between the training labels and the test labels.
Re:A popular self-driving car dataset is missing labels for ...
by bo1024 on Tuesday February 11, 2020 @05:06PM
(#22303427)
Strongly disagree. ML is traditionally deployed in low-risk scenarios like image labeling or recommendations where 1% or 5% error rate is totally "routine" and acceptable. Doesn't mean this kind of ML pipeline can translate to cases where errors are literally fatal.
This was a huge paper a couple years ago that demonstrates deep nets will still find the structure in the data even with totally randomized labels: https://arxiv.org/abs/1611.03530
There are also various well known techniques to verify whether you've over or underfit.