You can do this sort of stuff when replacing a web service too, by the way. For example running two versions of Django and checking if the new version produces any difference for a week before making it the version the client actually sees.
You can look at your code and say "For all invalid XML, this, for all input spaces, that." You can formally prove your code in other words.
You CANNOT do that with Neural Nets. Any formal proof would simply prove that your neural network simulation is still running okay. Not that it is generating the correct results.
You can supervise the learning process, and you can practically guarantee all the cases within your training data set, and everyone in the research space is comfy enough to say "yeah, for the most part this will probably generify" but the spectre of overfitting never goes away.
With machine learning, I developed a rule of thumb for applicability: "Can a human being who devotes their life to the task learn to do it perfectly?"
If the answer is yes, it MAY be possible to create an expert system capable of performing the task reliably.
So lets apply the rule of thumb:
"Can a human being, devoting their life to the task of driving in arbitrary environmental conditions, perfectly safely drive? Can he safely coexist with other non-dedicated motorists?"
The answer to the first I think we could MAYBE pull off by constraining the scope of arbitrary conditions (I.e. specifically build dedicated self-driving only infrastructure).
The second is a big fat NOPE. In fact, studies have found that too many perfectly obedient drivers typically WORSEN traffic in terms of probability to create traffic jams. Start thinking about how people drive outside the United States and the first-world in general, and the task becomes exponentially more difficult.
The only things smarter than the engineers trying to get your car to drive itself are all the idiots who will invent hazard conditions that your car isn't trained to handle. Your brain is your number one safety device. Technology won't change that. You cannot, and should not outsource your own safety.
I mean what would have happened here if another Tesla or two were directly behind Huang, following his car's lead?!
Possibly nothing, I'd assume the stopping distance would be observed and the following cars would be able to stop/avoid, but I wouldn't like to bet either way. Perhaps, in some conditions, the sudden impact on the lead car would cause the second car to lose track of the rear end of the first? Would it then accelerate into it?
An attentive human would realize something horrible had happened and perhaps reacted accordingly. A disengaged or otherwise distracted one may not have the reaction time necessary to stop the system from plowing right into the situation to make it worse.
An ML solution stops "learning" after training and only reacts.
To illustrate the difference, have you driven on roads under construction lately? As humans, when you've driven the same road hundreds of times, you start to do the same thing as a machine learned implementation. You drive by rote.
When you get to that construction zone though, or the lines get messed up, your brain will generally realize something has changed, and you'll end up doing something "unpredictable", i.e. learning a new behavior. The Machine Learning Ali's output (a neural net) can't do that. It can generify a pattern to a point, but it's behavior in truly novel circumstances cannot be assured.
Besides which, the problem still stands that the system is coded to ignore straight ahead stationary objects. Terrible implementation. It should look for overly fast and uniform increase in angular field coverage combined with being near stationary in terms of relative motion as a trigger to brake. I.e. If a recognized shape gets bigger at the same rate on all "sides" whilst maintaining a weighted center at the same coordinate. It's one of the visual tricks pilots are taught to avoid mid-air collisions.
Admittedly though, the human brain will likely remain WAY better at those types of tricks than a computer will be for a good long time.