After all, every driver on the road today is an incomprehensible black box where not only do we not know the parameters, we don't even know the function they're parameterizing. Every instance functions differently, and our testing procedures have woefully low coverage.
Not to mention that most software fixes cause other bugs...
We have precedent for how we qualify and evaluate things for safety: test them across a variety of conditions, accumulate driver-miles or operator-hours and incident frequencies. Then, using that data establish a bar for what constitutes an acceptable level of risk given the utility something provides. If we wanted to ensure nobody ever died in a car accident, we would ensure there were no cars, but collectively we've made a different choice.
Shutting down a plane is completely different from taking an entire class of publicly owned vehicles off the road. People will be furious.
Yes, they will be furious about the deaths and the shutdown, both. Don't forget that people are made up of individuals.
> entire classes of vehicles until the problem is confirmed fixed?
Yes, a malfunction AI would have to be grounded, just like for example the Boeing 737 MAX is now.
Yes of course we will. What is the problem with that approach? That is the exactly logical thing to do and will be done.
That's an extreme example, but automotive suicides that kill other passengers, drivers, or pedestrians fall into the same category. Consider also deaths from accidents involving drunk driving or fatigue -- thousands of motorists take to the roads every day modified in one manner or another that reduces their driving aptitude.
Also, while it may be correct to say that computers don't "fear death", there's no reason that "risk to self" can't be part of the criteria for decision making by an autonomous system.
Remember when the Toyota had that problem of the accelerator "getting stuck" because the software didn't disengage? Initially the owners' insurances were paying out, until it happened enough that they were able to prove it was Toyota's fault, and then Toyota had to pay them back.
I imagine in a self driving world it would work the same way. You get insurance, the car has a crash, your insurance and the manufacturer fight out whose fault it is.
The insurance will work much better than it does today, because insurance in it's core is about spreading the risks and calculating exact costs of those risks, it's about calculating statistics of negative events and predicting total costs of such events for the entire fleets.
ALL parts of that equation are just better calculated if all cars were automatic, - you can better calculate number of accidents, you can see details of all accidents because there is blackbox data including videos, you can compare cars to each other because a Tesla with same hardware drives in exactly the same way as another one (which cannot be said for human drivers), they don't have to calculate for weird human risk activities such as drinking or being tired, they can run simulations of the same situation on the same software etc. etc.
Insurance is not going to have any problems, insurance is going to love it and make a lot of money on the self-driving cars, they are a perfect fit for each other. Insurance companies don't even care for whom do they have to pay to, they just care that the statistic of the number of failures is correctly represented and that manufacturers don't lie about those statistics - that is all they care about, they calculate a simple equation, that's all insurance is about...
And if you go with the “nobody will own cars, you’ll just summon one” model... well the fleet owner will just sue the manufacturer instead.
For me? I'm a self-driving skeptic, but... if the manufacturer was willing to properly insure it, (I mean, a reasonable amount of insurance, at least a statistical life worth) I'd ride in the thing. I think that's an honest signal.
Its not just the manufacturers, who is underwriting all that insurance?
Ford sells approximately 2.3M vehicles per year, imagine if 50% of self-driving...over a 5 year period that 5.5M cars...if each one needs to carry a potential 1M policy thats an incredible amount of liability on someone's balance sheet. (even if you say the policy is only 100K thats still $576B in liability)
Thats only for Ford, add in all vehicles manufacturers and extend that to 10-15 years into the future and thats an incredible amount.
However there is nothing to say that a new laws won't be passed to allow manufacturers to escape liability. Most likely this is what will happen (see vaccine courts, etc)
You don't understand the complex weighted probabilities in your doctor's head either, but you trust them to diagnose cancer (which incidentally machine learning is beating humans at). None of the algorithms in doctors' heads can be formally proved to work in all circumstances, nor can the code that runs medical equipment.
A full understanding of complex systems (machine or human controlled) is not possible today in many domains, that's why we measure results. If the data shows that self-driving cars are safer, we will switch. At present, that's what it shows.
As to special roads/markers, these would make the technology less effective at dealing with the unexpected (crash ahead, moose on road, cyclist in the lane etc), and many of the leading companies don't think they are necessary. I can see cars forming networks which report danger, or adding more sensors, but don't think our roads will have to change for self driving, which will be prevalent within the decade IMO without infrastructure changes.
It is layers of redundancy. If one fails, the car continues to operate normally. If all are operating at peak, the car is near perfect. If multiple fail, it operates with somewhat degraded performance, but still markedly better than a human.
* Digital maps * P2P Networks * Human-reported obstructions and changes (Waze) * Machine-focused traffic markings * LIDAR * Cameras
Autonomous vehicles will only ever truly exist upon infrastructure literally designed to aid them, greatly simplifying how they need to interact with the environment, thus making the problem tractable with code we can prove works. I really think it will take more than putting markings on existing roads. It is going to take new roads full stop, probably with various wireless checkpoints built into them.