Re:Launch HN: Aquarium (YC S20) – Improve Your ML Dataset Qu...
by mlthoughts2018 on Monday July 13, 2020 @04:01PM
(#23824971)
I like this much better than synthetic data augmentation actually. I think synthetic augmentation, like with GANs is actually a failed concept.
There has long been theoretical limits around how much you can gain by ensembling with a model of known limitations, and this is all that synthetic training data is at root.
You can’t “make up” training data that allows you to escape the ceiling of performance implied by whatever generator process you use for the synthetic data, no differently than you can’t learning a better regression just by bootstrapping a large sample of data from your existing training set.
Algorithmic synthetic data is a big type of fool’s gold.
Said another way: once you've found "I do badly on green cones," we use similarity search on the embeddings of known green cone examples to find more instances of green cones in the wild. We pick the right examples from streams of unlabeled data, then send it to labeling + add to your dataset so it does better the next time you retrain.