One can be succeed at one level of using language without succeed at another level. But we humans expect another human to succeed or to fail fairly uniformly - or we call them "a bullshit artist". These expectations may not be met with large language models.
In that sense language models have a somewhat complete training dataset. Yes this requires an understanding of the material, rather than just parroting, but the signal is there if you can separate it from noise.
--> I'd agree that there's conceivably more that can be done with a language dataset. But the training process of transformers-based models isn't really oriented to engaging in the process you describe. It treated language as data and it's fundamentally a very sophisticated database that only appears to engage in such logic. As it will tell you.
By contrast it feels like driving requires an understanding of effects that are really hard to distill from pure driving data. Not just analytically hard, but requiring an understanding of external context.
But it's certainly a fine point.
For language, a problem to consider for a language using system is "talking a person through" a task. The thing about that is it involves two entities sharing a common model of reality and each updating their model as they listen to the other. And here I think corner cases of reality are basically as likely to show up.
Yeah, it struck me that perhaps a better approach to self-driving would be the road infrastructure and all the vehicles using it cooperating to build a local activity map - that way each individual vehicle doesn’t have to detect, classify and route around every object in real time, most of the work will have been done by previous vehicles and / or stationary traffic cameras.
Perhaps one difference is that a human could potentially get extremely good at textual tasks with nothing but text to learn from. You can read how to solve cryptic crosswords, see examples and extrapolate. In that sense language models have a somewhat complete training dataset. Yes this requires an understanding of the material, rather than just parroting, but the signal is there if you can separate it from noise.
Driving a car requires an understanding of a much wider context which is perhaps hard to acquire with just driving data. Understanding of rain, birds on the road, shaky drivers, balls rolling out from between cars, lane restrictions... You can't just throw petabytes of data at the problem. Training data is limited and expensive, and I believe we are mostly tackling AI-assired driving with rule-based approaches.
I believe self driving works just fine in simulations where data is effectively unlimited. But then it doesn't generalise to the real world where context matters.