An interesting analog is the history of spin training in the US. Until the 1950s, a pilot was required to enter and recover from a 3 turn spin on their private pilot checkride. The thought was spins are dangerous, so pilots should show they can recover. The problem is that spins ARE dangerous, so a large number of students and instructors were spinning into the ground during training.
The FAA decided to remove the spin from the checkride. This effectively removed it from training. Many instructors lamented this, claiming the spin accident rate among private pilots would increase dramatically. They couldn't be expected to recover from something they had never trained for. The accident rate did change. It dropped sharply. But why?
The first part is the FAA accomplished its goal. Less pilots intentionally spinning meant less pilots spinning into the ground. But this didn't explain the drop in spin related accidents in nontraining situations.
It all came down to the airplanes. The airplane that most pilots train in is the same kind of airplane that most general aviation pilots fly: small, slow, single engine two or four seaters. Airplane manufacturers designed airplanes that could fulfill the needs of both the student pilot and weekend pilot. This meant a school could rent out its training airplanes, and a pilot could learn in the airplane he meant to own long term.
One of the requirements of the student's airplane was that it could be used for spin training. This meant making an airplane easy to spin. The problem here is clear in retrospect. You have a population of weekend pilots flying airplanes that are specifically designed to spin easily!
Removing the training requirement removed the manufacturing requirement. There is a clear difference in the design of airplanes before and after this change. Compare the Cessna 140 and 150, or the Piper Super Cub and Piper Cherokee. The later model is incredibly more difficult to spin than the earlier one.
Spin accidents in these post change airplanes are almost nonexistent. The FAA statistics are even misleadingly negative. They lump stall/spin accidents together. But, if you look into the accident reports in detail, you'll find a fully developed gyroscopic spin into the ground is incredibly rare.
This story can serve as an example for other interactions between regulations, operators and machine. Remove training requirements that do more harm than good. Focus on emergency avoidance instead of emergency recovery. Allow engineers to create the safest possible design, even one that doesn't pass archaic requirements.
No pilot, ship captain, or driver operates in a vacuum. If we want the best possible environment, we have to look at the environment as a whole.
That's also partially why we had Asiana 214 - if the pilot at the controls was more skilled at hand flying the aircraft he would not have been so dependent on the autothrottle functionality to manage the speed of the airplane.
I worry deeply about creeping automation not only on the flight deck but in our cars. Although automation is far more responsive and adds greatly to safety in both cases, dependence on automation will breed a new generation of the "Children of the Magenta" [2] who blindly allow the computer to follow the magenta line on the display, either into the ground in an airplane, or into oncoming traffic in a car as they ask themselves "what's it doing now??!"
I saw an interesting panel last year at the MIT Sloan CIO Symposium where Missy Cummings from this lab (who used to by a navy fighter pilot) talked about a number of these issues.
Imagine if self-driving cars were at a point where they were 10% better (in terms of fatalities caused) than human drivers. You start flooding the roads with them, and fatalities decrease.
Then the news stories start. "Mother of Four Killed by Self-Driving Car." Video of the grieving children. News shows host "experts" who go on at length about the deficits in computer judgment. The family sues, with their lawyer saying that this death would not have happened with a human at the controls. (And he might be right! Even if better than humans, computers will probably get into different accidents, not just fewer.) Restrictive regulations or even bans start being discussed in Congress.
Maybe it wouldn't happen this way, but it could potentially set things back by years. Looking at the technology, pushing for early adoption might save lives. But looking at the politics, pushing too hard could cost lives, by making people unwilling to accept the technology.
It would absolutely happen this way.
Like any change, driverless cars would cause a lot of people to lose money and/or their livelihoods. So there will be a significant number of people (and businesses) who fight against the change.
Look at climate change -- the actual metrics don't even matter. There is always a way to skew the display of the numbers to make a pretty graph that will convince enough of the population that there's at LEAST some legitimate controversy over the issue; which is enough to just put the decision off until the controversy is over.
Such a car would be immensely safer than what we have now. Would people willingly ride in one?
I think I'm pretty good at rationally weighing the risks, but just imagining this still gives me the willies.
I see not only Google, but a number of auto manufacturers, doing a lot of work. The Google testing, which is on a relatively small number of precisely mapped roads in good weather, has had a quite a few incidents that required a human to intervene (which won't be a realistic option once normal drivers turn over control). So from my perspective there's quite a bit of work going on but a lot more experience and work is needed before this is something that can be sold to consumers even for limited use cases.
(What we are already getting in high-end vehicles is assistive driving systems which should reduce accidents as they become more widespread--at least so long as they're not so effective as to increase issues associated with inattentiveness.)
Since human pilots (on commercial flights) are either discouraged or prohibited from actually flying (for legitimate safety reasons), when the computer does experience a failure, the human is unskilled to take over. Yet, ironically, it appears that if we encouraged the human pilots to hand fly more, the accident rate would probably increase.