Re:Visual Introduction to Self Supervised Learning
by trash3 on Friday February 28, 2020 @05:14PM
(#22446904)
So instead of image annotation, self-supervised learning performs image manipulation to train a model. Then what? Is this network then piped into the original task at hand which would have required human annotations or is it simply for these made up tasks?
[ Parent
| on HN ]
Re:Visual Introduction to Self Supervised Learning
by Voloskaya on Friday February 28, 2020 @05:45PM
(#22447121)
You then add a few additional layers on top, and you train those new layers in a classic supervised way.
But because a lot has already been learned you need way less labels.
Re:Visual Introduction to Self Supervised Learning
by ivalm on Saturday February 29, 2020 @02:06AM
(#22449384)
Two main things you can do:
1) Transfer learning -- start with self supervised model and either fine tune parameters or freeze parameters + add another layer to train your task (with way fewer params/necessary labels since you already learned about the input distribution)
2) Nearest neighbor/clustering -- no need to label all classes, simply fetch similar examples (eg find semantically similar sentences in a corpus).
Re:Visual Introduction to Self Supervised Learning
by contravariant on Friday February 28, 2020 @08:03PM
(#22447883)
Optimistically if a self-supervised algorithm is capable of understanding a concept then it shouldn't need all that many examples to make it useful. Ideally you could just show it what cats look like (with just a couple of examples) and ask it to find more of them.