I just want to see "the vehicle again alerted the driver to put his hands on the wheel" followed by "and Autopilot then pulled to the side of the road, turned on the hazard lights, and required the driver to take over."

>and Autopilot then pulled to the side of the road

"It's a winding road going through a canyon, with no shoulder"

So going from a dangerous situation to another dangerous situation.

That's stupid. Autopilot doesn't have the ability to determine if it is safe to pull over. It has a fallback mode; eventually it will come to a complete stop in the middle of the road and turn the hazards on. But since this is also dangerous it makes sense to give the driver time to stop it from happening.

It's stupid to come to a gradual slowdown with hazard lights, but it's not stupid to keep blazing away at speed, with an obviously inattentive (or perhaps completely disabled) driver? I'm confused - even you acknowledge it has that fallback, but it's 'stupid' to enact it after a couple of failed warnings? When should that threshold be crossed, then?

That is exactly what it does; apparently the accident happened before it could complete its "slow down and stop with the hazard lights on" process.

so far we have been lucky the accidents have only impacted the driver. is it going to take the first accident where an autopilot car kills another innocent driver before people see the problem?

When autopilot kills more innocent drivers than other drivers you can point me to a problem. Time will tell if it is better or worse and the track record up until May/June was going pretty well.

I'd rather 10 innocent people die to freak autopilot incidents than 1,000 innocent people die because people in general are terrible at operating a 2,000-3,500lb death machine. Especially because everyone thinks of themselves as a good driver. It's fairly obvious not everyone is a good driver - and that there are more bad drivers than good.

Maybe I only see things that way because there have been four deaths in my family due to unaware drivers which could have easily been avoided by self-driving vehicles. All of them happened during the daytime, at speeds slower than 40mph, and with direct line of sight. Something sensors would have detected and braked to avoid.

Assuming we stick with the same conditions as in this case, I'd call an innocent life taken due to a driver disregarding all warnings from his system negligence. That also fits with what this driver was ticketed with which was reckless endangerment. We don't even get heads up display warnings about not driving into oncoming traffic, but nobody is going to blame a dumb-car for the actions of its driver. I would say that inaction in this hypothetical case would be the driver's crime.

The Tesla warnings remind me of Charlie and the Chocolate Factory, where one of the brats were about to do something disaterous and Gene Wilder would in a very neutral tone protest "no, stop, you shouldn't do that".

From a risk management standpoint, Tesla's design doesn't strike me as sufficient. Warnings without consequences are quickly ignored.

The car crashed. Sounds like a pretty strong consequence to me.

I guess I just don't know how many flashing warnings somebody has to ignore for it to finally be their fault.. At some point, their unwillingness to change their natural inclinations in the face of overwhelming evidence that they need to should have a negative consequence. This isn't like "just click through to ToS". If you buried it in the ToS or something similar, I absolutely agree with your point of view, but when you make it painstakingly clear in unavoidable manners I think we've crossed into territory of user responsibility. If this feature needs to be removed from Tesla cars (and I accept that potential future) I would definitely put that "fault" on the users of Tesla cars rather than Tesla based on the implementation I've read about.