We accept that human drivers kill thousands upon thousands of people. We won't tolerate deaths from automation even in the single digits.

That may not be rational but it's also unlikely to change.

The rational fear is of automation introducing systemic failures. Imagine a "flash crash"[1] type event applied to widely deployed self-driving vehicles.

[1]: https://en.wikipedia.org/wiki/2010_Flash_Crash

I never understand why it has to be either one or the other. Either we accept deaths caused by human drivers, or we go full autonomous and we accept deaths by software errors.

The way I see it, AI assisted driving is the way to go. Fully autonomous driving is, at least in my opinion, a pipe dream. No amount of testing will ever be able to prove an AI is safer than a human in all situations. They might be safer in some though, e.g. highway drives, quiet roads at night etc. So instead of pouring in vast amounts of money developing fully autonomous vehicles, why not spend it on improving the existing AI assistance features that already exist, so we can prevent most human errors, without introducing new AI failure modes.

Note that this means we should all forget about the dream of not having to pay attention on the road because 'the car drives itself', and that we should actually dial back things like Tesla autopilot, which venture too close to 'car drives itself'. The driver should be in control, until he/she makes a mistake, then the AI should prevent worse.

On a side note: people comparing plane autopilots to point out the merits of fully autonomous driving are really missing the point, there is no AI in avionics whatsoever.

I do think we'll get to fully autonomous for some significant useful subset of driving such as paved public roads and can probably do things to help out the AIs with corner cases (beacons, etc.). That said, I actually agree that the last 10% that gets things to reliable Johnny-cab level are probably a lot further than many are assuming. Given the level of investment and advances that have been made I'm probably even more optimistic than I was, but I still think it could be 30-40 years before you can order a robo-Uber in Manhattan.

Yet, Robots and automated machines (not the autonomous vehicles) still killed/injured a few dozen people since 1984 according to U.S. Department of Labor. [1]

[1]: https://www.osha.gov/pls/imis/AccidentSearch.search?acc_keyw...

Worldwide stats will be far more than that. We've accepted it, haven't we?

They're freak accidents. We wouldn't tolerate deaths day in and day out to the tune of 40,200 in the US in 2016.

..too lazy to find and paste stat from mobile

You said we won't accept those deaths "even in the single digit", not 40k+ deaths, which is a few orders of magnitude higher.

Also I don't understand why we would accept freak accidents but not other accidents? An accdient is an accident. A death is tragic regardless what caused it.

We don't accept the robot deaths. Any time one happens people scramble to "make sure this never happens again". When someone dies in a car accident, we yawn.

I think it's a matter of sussing out liability.

Will you find acceptable a plane crash because of technical failure?

Determinism rules for aeronautical engineers (software and hardware), and thus, today, flying is the safest transportation mode.

> Will you find acceptable a plane crash because of technical failure?

Oh but they did and still do, but has improved, as every technology matures. My great-grandparents traveled a lot by airplane in the 50's, and always traveled separately so the kids didn't lose both parents in the event of a crash, but guess what happened the one time they traveled together...

> It is estimated that approximately 22 percent of aviation accidents are caused by mechanical failures.

Source: http://www.trial-law.com/aop/mechanical-failures/

Whoa, did they both really die together in a plane crash?! How tragic if true.

> The exact same can be said about human drivers

Maybe not. There are cautious drivers and there are rash drivers. The probability of accident is wildly different for both of them. However the probability of an accident due to a a bug/miscalculation in autonomous driving is same for everyone.

And when that bug is fixed, assuming modern update practices, it is fixed for everyone. In effect, human drivers can only learn from their own mistakes whereas autonomous systems learn from every mistake made by their system's drivers.

But can you really fix a bug in a machine learning algorithm? Sure, you can retrain it on the data which caused the crash, but could that mean you created another bug somewhere else?

That's exactly why I am not looking forward for self-driving cars. Rash drivers in most cases do know they are doing something stupid and dangerous (like going way over speed limit, overtaking where visibility is limited, etc.) AI on the other hand is prone to "honest mistakes" which can be deadly.

They shouldn't be because even as a cautious driver you are subjected to noncautious drivers and, more importantly, no matter how cautious you are a well designed AI is almost certainly better.

Both cautious & rash drivers face the same external factors, but latter are more likely to wreck themselves, sometimes without involving others. So probability of an accident for latter are higher.

AI may be good at reducing the average number of accidents, but when a tricky situation occurs, the probability of accident could be in this order "rash > AI > cautious".

To be honest, i had the same assumptions on cautions and rash drivers. But then i learned to know my new boss- he drives ralley-race cars- and to be honest, i feel much safer in such a ralley car taking to its physical limits by someone who knows the car- then in the car of a cautious driver, who is just cautious because he/she is aware of how little they know/controll the actual physical event of driving a car.

I know such a thing is nont quantifiable thus, the laws min(ability) applies, none the less, one should not judge from appearance here, but rather from actual incidence lists.

Liability for an accident not related to equipment malfunction generally stops with the human driver. Liability for accidents related to autonomous systems will not to be so limited.

To err is human. We can punish erring drivers by law. But we cannot punish a robot. Unless there is law that applies to the owner of the autonomous vehicle companies on erring automatons unleashed by them :-) and they are implementable.

Do you punish a human because of payback or so that it doesn't commit again the same error ?

Of course its revenge. A eye for a eye, a tooth for a tooth. You are not talking with a civilized species here, although the philosophical decoration sometimes can lead one astray.

We humans aren't rational beings. We're rationalizing beings. I would even suggest we've always felt ourselves to be the epitome of enlightened - even whilst committing atrocities.

There are results showing that machine learning can be completely fooled by images that are slightly modified, so little that the human eye can not perceive any difference. Have a look at the images in this paper on "adversarial examples": https://arxiv.org/abs/1412.6572

let's assume that neither humans or machines are perfect drivers. but there is a non-zero probability that the machine can do something nonsensical that a sane human would never do (due to some unknown edge case). who do you trust more to be your driver?