This is the only rationale I can come up with for Tesla's approach. The tenets go like this:
1) The ultimate realization of self driving will be successful, and will have profoundly positive impacts on society (e.g. safety, urban planning, ease and availability of transportation, economic)
2) The path to that ultimate realization contains many local optima traps
3) The only way to push past those local optima is by acquiring large amount of real world automated driving data, using systems in production
From a technical standpoint, I'm disinclined to disagree with them on the necessity of large amounts of real world system behavior data as a key ingredient.
Personally, I also think we (humans) have become way too culturally risk adverse. Or, to put the calculus another way: if I have a credible belief that I can save some number of future humans, what is the acceptable moral number of current humans to place at risk and/or kill?
Imho, that's a grave, serious decision, but not one to which the morally absolute is "zero, always."
From a purely ethical standpoint (leaving legal aside), the crux would seem to be informed users. Not all Tesla drivers are likely capable of understanding the current SotA, and I'd hope (and expect) that Tesla's phased rollout of the beta is informed by previous user interaction with assisted driving.
> Imho, that's a grave, serious decision, but not one to which the morally absolute is "zero, always."
I think that what we've learned throughout human history is that no one should be trusted and given the power to make such decisions. It has invariably led to disasters. The very idea that someone can decide to put someone else's life in danger for potential future advances gives too many perverse incentives - chief of which, rationalization that "more people will be saved" even as the experiment crashes and burns.
When it's for the greater good, or by the winners, we conveniently omit it from history.
When it's for the greater evil, or by the losers, we publicize it as a cautionary tale of moral hubris.
Example 1: Vaccines
This is an interesting example because the winners and losers seem to be proportionately spread across almost all of society (except anti-Vaxxers who benefit from herd immunity but don't sacrifice with the minute risk of vaccines). See also: https://www.hrsa.gov/vaccine-compensation/index.html
Example 2: War on Crime / Safety
Another example would be the war on crime. Liberties and lives are sacrificed in the name of safety for others. Unfortunately, the liberty/lives sacrificed fall disproportionately into one group, and those protected fall disproportionately into a different group group. If you look at the 2nd order effects (impoverishment, single-parent households, effects on education and future) lots more lives are lost, again in a subset of society.
This is clearly possible in principle. After all, fully autonomous vehicles can have much more capable sensor systems than any human driver, can process the input from those sensors faster, and can then apply any necessary adjustments to the vehicle's behaviour almost instantaneously, all without ever getting tired or distracted. No human driver will ever match that potential.
The problem I have with this whole argument, and thus the reason I'm not sure the moral/ethical arguments stand up either, is that we don't know how to reliably write excellent software yet.
The entire argument for autonomous driving rests on the premise that we will at some point in the usefully near future reach a situation where the cars drive themselves much better than we humans collectively do. But there is, to the best of my knowledge, no evidence so far that we as a society can actually create the necessary software yet.
Until we can, any autonomous vehicle control system will be at risk of life-threatening bugs. In particular, it will be at risk of widespread failures of the same type that could cause much more harm than any single human driver's failure could, whether caused by defects in the software alone or by a malicious actor exploiting a security vulnerability. That is a huge risk that is getting far too little attention so far IMHO.
The first is whether we expect the autonomous vehicles to become safer than the population of human drivers they would be replacing under normal circumstances. This is a population measure, and it's about overall safety and averages, and given enough time and data it seems a realistic outcome.
The second is the risk of a single catastrophic failure, which is a risk that doesn't generally exist with today's human-controlled vehicles. It is no exaggeration to say that the kinds of autonomous vehicle technologies we are conceiving here have the potential, if they go wrong, to cause more harm than any WMD ever deployed in war. And not only the theoretical potential, but given the current state of our software development abilities, a non-trivial chance of it actually happening in practice.
Until we've dealt with the latter problem, which I don't believe we currently know how to solve, any arguments based on the long-term superiority of autonomous vehicles over the capabilities of any human driver is going to be fundamentally flawed.
This is what it really comes down to.
Imagine if we invented the car today in our current cultural climate. I don’t think it would be possible for humanity to reap it’s advantages.
Back when it was invented people actually had to drive the things around on public roads that had a bunch of pedestrians walking haphazardly accross them continuously.
I was an autopilot skeptic, and it’s entirely for this reason. I didn’t think it would be possible for anyone to actually do what needed to be done to get this technology across the line. That is to say, give it to customers before it is safe.
If Tesla is actually able to keep doing this without it getting killed politically, then I think it actually will happen, and that gives me some hope for humanity more generally.
That, I don’t see ever being true. What would you get from that that you couldn’t get from “large amount of real world human driving data, using systems in production”?
Or to put it another way, every time the driver's inputs diverge from the automated system's inputs, the test run is invalidated. You're no longer getting results of what the automated system did.
In Germany they lost a case about their Autopilot advertisement. So not even legally they are in the clear.
Legal : If you sell it as a drive assist and not a driver replacement, you're off the hook, with a high probability. It's on the drivers if they get complacent. There may even be precedents with similar products for trucks.
Moral: pushing this now will very likely kill some people that would otherwise be alive (drivers or pedestrians). But it's very unlikely to be a carnage - I'd dare say even impossible. On the other hand, moving the technology forward and directly into widespread commercial use will save people, and likely even... what's the equivalent word for a "carnage" but where people are saved? Miracle.
Branding: well, there you have a real risk. Public opinion is fickle, and a particularly gruesome accident might do some real damage, both to Tesla and to self driving technologies in general. But then, they do have a high risk high reward mentality, and the chance is good that the first bad accident (statistically inevitable) will come after the advantages are proven and well established.