That may be what you extracted from this incident, but it's definitely not "the lesson".
Tesla should improve its systems regardless of whether it was the first such error, the seventh, or the tenth.
But mostly... I just don’t understand it. It seems like such an odd decision by the driver I don’t know what to make of it.
It’s very unfortunate that he died, but that doesn’t mean we can’t ask questions/criticize what may have gone on.
But he wasn't operating it.
The expectation that the driver will jump in to prevent a car's autopilot from steering the car off the road into an obstruction is ridiculous. It's one thing to expect the driver to stop for on-the-actual-road dangers. This wasn't.
We don't really know what happened yet, but if it was the autopilot and it did steer into the barrier, this is at least 95% Tesla's fault.
As we see in latest Tesla report, there were multiple warnings to which the driver should respond: this means, that the problem was detected with enough time to react and the autopilot was not able to solve it on its own. Thus, it does not make sense to blame the autopilot - the technology did behave as expected (when problem cannot be solved, report asap to the owner, that he should intervene).
What Tesla can actually be blamed for, is the misunderstanding of the autopilot capabilities by the drivers. It's not an algorithmic bug, but rather a usability one. Perhaps, the warning should be issued earlier or in a more clear way. Perhaps, the drivers should receive some training, to act as supervisors of the system, not as passengers, or there even should exist a special driving license for such types of cars.
Nevertheless, the main fault was not the driver's and not the Tesla's. The safety of this road should be guaranteed by the authorities in charge, the navigation should be made sufficient for the drivers, and they clearly failed to do so.
Seriously though....I doubt it. The report said he was told to put his hands back on the wheel "earlier in the drive". I seriously doubt he would have had time to react in this case. This is not the same as a plane...there was no "ground alert" or anything like it. From what I can tell, it was doing its job then steered right into a wall. That sort of thing simply doesn't happen with a plane....when autopilot is active on a plane, there is never a situation that requires instantly noticing something is wrong, and correcting it, within a few seconds. Never.
And usability bugs, if that is what you want to call this, are indeed Tesla's fault. They, not the driver, have the resources to test and understand how human attention works (i.e. "vigilance deficit" / "handoff problem")
In a commercial jet - Goes to the toilet? Eats lunch? Chats to the co-pilot?
Sure none of those might be right but I'd warrant that a lot of people would answer with things that move the pilots concentration away from piloting, "because the auto-pilot is doing the flying". Honestly I'd assume they do any of these answers; but with the proviso that the co-pilot stays in the cockpit when the pilot leaves.
Of course people may have abused the system if it had a more sensible name anyway. But names competitors use like Automatic Cruise Control or SuperCruise seem much more descriptive of the system’s actual abilities.
Would it make a difference if you learned that the driver knew there were problems with autopilot and was using it anyway?
Somewhere between 7-10 iterations of the same dangerous failure mode, it's time to stop betting your life that the failure won't repeat.
> Walter complained "7-10 times the car would swivel toward that same exact barrier during auto-pilot."
This still doesn't diminish Tesla's obligation to make their autopilot system behave properly.