News for hackers. Stuff that matters.
Re:Visual Introduction to Self Supervised Learning by trash3 on Friday February 28, 2020 @05:14PM (#22446904)
So instead of image annotation, self-supervised learning performs image manipulation to train a model. Then what? Is this network then piped into the original task at hand which would have required human annotations or is it simply for these made up tasks?
Re:Visual Introduction to Self Supervised Learning by Voloskaya on Friday February 28, 2020 @05:45PM (#22447121)
You then add a few additional layers on top, and you train those new layers in a classic supervised way. But because a lot has already been learned you need way less labels.
Re:Visual Introduction to Self Supervised Learning by ivalm on Saturday February 29, 2020 @02:06AM (#22449384)
Two main things you can do:

1) Transfer learning -- start with self supervised model and either fine tune parameters or freeze parameters + add another layer to train your task (with way fewer params/necessary labels since you already learned about the input distribution)

2) Nearest neighbor/clustering -- no need to label all classes, simply fetch similar examples (eg find semantically similar sentences in a corpus).

Re:Visual Introduction to Self Supervised Learning by contravariant on Friday February 28, 2020 @08:03PM (#22447883)
Optimistically if a self-supervised algorithm is capable of understanding a concept then it shouldn't need all that many examples to make it useful. Ideally you could just show it what cats look like (with just a couple of examples) and ask it to find more of them.