> My hypothesis is YES. If you train a HUGE image model using vast quantities of raw images, you will then be able to REUSE that model to work for specific computer vision problems, either by fine-tuning or 0/1-shotting.
This has been demonstrated for many years, it's not news. Many of the SOTAs like BiT require pretraining on JFT-300M, or Instagram, or what have you.
The pretraining approach was used in vision for years before it was successful in NLP.
The crucial question is : is this paradigm viable for OTHER types of data?
My hypothesis is YES. If you train a HUGE image model using vast quantities of raw images, you will then be able to REUSE that model to work for specific computer vision problems, either by fine-tuning or 0/1-shotting.
I'm especially optimistic that this paradigm will work for image streams from autonomous vehicles. Classic supervised learning has proved to be difficult if not impossible to get to work for AV vision, so the new paradigm could be a game-changer.