Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same results.
It has also been my primary focus for the past 5 years and the core of the company I founded.
And then, someone is using pretrained 500B model, and fine-tuning your few examples, and getting new SOTA.
Simplest way to weigh by sample efficiency: multiply accuracy by ratio of test set to training set sizes. Everyone's training/testing on 80/20 splits, so everybody's SOTA would go down by 3/4s.
Already in 2018 SenseTime reported that for face recognition, clean dataset surpasses accuracy of 4x larger raw dataset.
Only, the article seemed to show a very conservative Ng about the algorithms, a focus on data management - so it's still ML.
I also want cars that run on salt water.
I'm not saying that small data ai is equally impossible, but simply saying "we should make this better thing" isn't enough.