When one of those black boxes malfunctions it gets taken off the road. When the AI malfunctions, are we going to shut down entire classes of vehicles until the problem is confirmed fixed?

Not to mention that most software fixes cause other bugs...

In an extreme example I expect that's precisely what would happen. Consider what's currently unfolding around the 737 Max. In the automotive space there's a long history of serious flaws that resulted in loss of life, ranging from faulty airbag deployment systems to flawed designs for ignition systems.

We have precedent for how we qualify and evaluate things for safety: test them across a variety of conditions, accumulate driver-miles or operator-hours and incident frequencies. Then, using that data establish a bar for what constitutes an acceptable level of risk given the utility something provides. If we wanted to ensure nobody ever died in a car accident, we would ensure there were no cars, but collectively we've made a different choice.

We've made a choice to allow people to kill each other in cars from time to time, but that's different from choosing to allow automated cars to kill anybody. Knowing human nature, I don't think the general public will accept double digit automated deaths without an outcry.

Shutting down a plane is completely different from taking an entire class of publicly owned vehicles off the road. People will be furious.

Yes, they will be furious about the deaths and the shutdown, both. Don't forget that people are made up of individuals.

> When the AI malfunctions, are we going to shut down

> entire classes of vehicles until the problem is confirmed fixed?

Yes, a malfunction AI would have to be grounded, just like for example the Boeing 737 MAX is now.

Or a car model with severe problems. This rarely (if ever) happens because with that many cars, severe problems tend to be noticed fairly quickly. That shouldn't change with self driving cars. With several million miles driven each day for more popular models, even rare edge cases should appear within days.

> are we going to shut down entire classes of vehicles until the problem is confirmed fixed?

Yes of course we will. What is the problem with that approach? That is the exactly logical thing to do and will be done.

At least the software is fixable. Other humans are not.

At least other humans fear death as a consequence of driving incorrectly. Computers do not.

Some humans, when seeking to end their own lives, end the lives of others: https://en.wikipedia.org/wiki/Germanwings_Flight_9525

That's an extreme example, but automotive suicides that kill other passengers, drivers, or pedestrians fall into the same category. Consider also deaths from accidents involving drunk driving or fatigue -- thousands of motorists take to the roads every day modified in one manner or another that reduces their driving aptitude.

Also, while it may be correct to say that computers don't "fear death", there's no reason that "risk to self" can't be part of the criteria for decision making by an autonomous system.

Too many people still driving around drunk sadly counters that point.

Humans are fixable in that they are held accountable. One person can be taken off the road if they are unfit to drive. (Revoke license, imprison, etc). Then the incident has become Someone's Fault, and society can move on.

I’d personally rather have less death and mayhem than be able to blame someone for it...

That's a fine opinion, but it's the minority. Maybe not in the abstract, but as soon as you have unexplainable deaths (meaning there's nobody to blame), people freak the fuck out.

how do you see insurance working? why would drivers be responsible to have insurance when the car is completely controlled by bigcorp programmers?

I have to have insurance now, even though a lot of the functions of my car are controlled by their software.

Remember when the Toyota had that problem of the accelerator "getting stuck" because the software didn't disengage? Initially the owners' insurances were paying out, until it happened enough that they were able to prove it was Toyota's fault, and then Toyota had to pay them back.

I imagine in a self driving world it would work the same way. You get insurance, the car has a crash, your insurance and the manufacturer fight out whose fault it is.

Sometimes it seems that critics of something like self-driving cars want so badly for the project to fail that they themselves fail to see obvious solutions.

The insurance will work much better than it does today, because insurance in it's core is about spreading the risks and calculating exact costs of those risks, it's about calculating statistics of negative events and predicting total costs of such events for the entire fleets.

ALL parts of that equation are just better calculated if all cars were automatic, - you can better calculate number of accidents, you can see details of all accidents because there is blackbox data including videos, you can compare cars to each other because a Tesla with same hardware drives in exactly the same way as another one (which cannot be said for human drivers), they don't have to calculate for weird human risk activities such as drinking or being tired, they can run simulations of the same situation on the same software etc. etc.

Insurance is not going to have any problems, insurance is going to love it and make a lot of money on the self-driving cars, they are a perfect fit for each other. Insurance companies don't even care for whom do they have to pay to, they just care that the statistic of the number of failures is correctly represented and that manufacturers don't lie about those statistics - that is all they care about, they calculate a simple equation, that's all insurance is about...

Insurers like standards and features that can be easily verified and improve predictability of the crowd. The issue with high tech solutions, especially mono-cultures is malicious hacking or outages of central services that result in simultaneous failure. An insurer can't handle 50% of cars crashing in the same year.

Insurance would be a nightmare for a manufacturer. Every accident will initially be pegged to the auto maker (as it should be.. it’s their code!). The auto maker will always try to weasel out and blame the passenger-owners of the car (they didn’t maintain it, the paint was dirty and messed with the sensors, the tire pressure was 2 PSI lower than average).

And if you go with the “nobody will own cars, you’ll just summon one” model... well the fleet owner will just sue the manufacturer instead.

Just like Tesla blames dead drivers for using "autopilot." "They should have kept their hands on the wheel and been paying attention." No you can't have a copy of the data.

Seems like it would mostly be the manufacturers that would have to insure the cars, at least for the expensive part (liability)

For me? I'm a self-driving skeptic, but... if the manufacturer was willing to properly insure it, (I mean, a reasonable amount of insurance, at least a statistical life worth) I'd ride in the thing. I think that's an honest signal.

>... if the manufacturer was willing to properly insure it,

Its not just the manufacturers, who is underwriting all that insurance?

Ford sells approximately 2.3M vehicles per year, imagine if 50% of self-driving...over a 5 year period that 5.5M cars...if each one needs to carry a potential 1M policy thats an incredible amount of liability on someone's balance sheet. (even if you say the policy is only 100K thats still $576B in liability)

Thats only for Ford, add in all vehicles manufacturers and extend that to 10-15 years into the future and thats an incredible amount.

However there is nothing to say that a new laws won't be passed to allow manufacturers to escape liability. Most likely this is what will happen (see vaccine courts, etc)

But all of those cars are insured (and that insurance is underwritten) today. So the liability already lives on the balance sheet of insurance companies. Maybe the specific companies change...

eh, right now most people are massively underinsured; minimum coverage in California is like $35k, and most insurance companies won't sell you a plan with more than a half million of liability (at least not without an umbrella policy) - if we stop subsidizing driving through pushing costs on to victims of accidents, the cost of driving will go up. But yeah, it should be about the cost of a good umbrella policy+auto policy is now, modulo any savings if the self-driving car gets in fewer accidents.

but note, we're paying most of that already in the form of people who are killed and under-compensated by under-insured drivers. Increasing liability insurance minimums would roll that cost that is currently born entirely by the victim into the cost of operating a car, which is where it ought to be.