Not sure I see how that would work when there is no 3G signal. If a computer on-board a vehicle sees something it does not recognize when it's not connected to the Tesla HQ, what should it do? And even if it is connected, uploading video over 3G is too slow for the real-time classification needs. Right?

The notion isn't that they phone home to immediately ask about unknown data, but that they periodically feed back any unknown data (or, in the worst case, it gets extracted from a blackbox if a car has an accident), and receive revised models to process the world with.

No, this won't necessarily save you if you see some impossible scenario like a boat cruising down the highway toward you, but it does mean that the list of conditions the model can't respond to reasonably will rapidly trend toward "fewer than most human drivers", in the ideal case.

This will, invariably, have some unfortunate bumps when people discover real-world conditions that, for whatever reason, the model doesn't remotely have responses for (I wonder if they've trained it on e.g. an enormous wall of water, or tornadoes?), but that's why you don't claim it's an always-on self-driving system (e.g. you have to be ready to take over at any point), and arguably the error rate is still going to be lower than most humans to start with.

You need humans to label those "millions of experiences." The bottleneck is not raw video. You need humans to label that data. Otherwise it's useless.

No, you don't. You need to process enough of it to see how the majority of human drivers act in situations where the automated system currently would react substantially differently.

You've just described end-to-end-learning (e.g. the data is raw video/camera/radar/sensor data, the labels are steering angles, etc.)

No autonomous vehicle manufacturer uses end-to-end learning. The only one to claim to use it was Comma.ai, and we all know how that went.

All the autonomous car companies will manually label the camera images - e.g. given an image, draw boxes around where all the cars are.

Learning how to label camera images and how to respond to already detected features are separate issues. You can make the choice of whether to apply unsupervised training to either one separately.

This [1] blog post from Tesla explicitly claim that they are using unsupervised learning as one of their strategies to determine appropriate system response to specific detected objects in specific locations:

> This is where fleet learning comes in handy. Initially, the vehicle fleet will take no action except to note the position of road signs, bridges and other stationary objects, mapping the world according to radar. The car computer will then silently compare when it would have braked to the driver action and upload that to the Tesla database. If several cars drive safely past a given radar object, whether Autopilot is turned on or off, then that object is added to the geocoded whitelist.

> When the data shows that false braking events would be rare, the car will begin mild braking using radar, even if the camera doesn't notice the object ahead. As the system confidence level rises, the braking force will gradually increase to full strength when it is approximately 99.99% certain of a collision. This may not always prevent a collision entirely, but the impact speed will be dramatically reduced to the point where there are unlikely to be serious injuries to the vehicle occupants.

[1] https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...