> I also think that many people don't completely understand how weak this model is to process real-world data when there no reliable way for the system to self-train on it.

True, but probably any data can be used in self-training if you have captured enough of it. ML models can handle sequences, grids, trees, graphs, point clouds and sets as input formats.

As for needing self-training in general, humans also need it even though we have such well attuned priors on the world based on billions of years of evolution. Even so, it takes years before a human can utter a phrase or accomplish a complex task.

Basically today ML works by interpolation, if you have enough data points to cover your space you're good. But if you want to extrapolate outside, you're screwed.

<rant>What we need is powerful simulators (physical or even neural based) to expose ML models to a variety of tasks inside the target domain. A simulator is like a dynamic dataset. A ML agent inside an environment (a simulator) is much more similar to humans than a model + dataset. Of course simulation is just as hard as AI.</>