Arguably. Tesla has not done a good job educating their customers on the limits of their "Autopilot" system, which has led to at least one preventable death.

https://www.theregister.co.uk/2016/07/28/tesla_autopilot_dea...

Is it possible to extrapolate from the recorded reduction in accidents after the introduction of autopilot and work out if fewer people have died as a result of introducing when they did?

Here's an article which describes a 40% drop in the accident rate:

https://electrek.co/2017/01/19/tesla-crash-rate-autopilot-nh...

The NHTSA's figures show a drop from 1.3 crashes per million miles before Autosteer to 0.8 crashes after Autosteer.

Would any of those crashes have been fatal? I don't have the numbers to answer that but I posit that introducing Autopilot has prevented more deaths than it has caused.

The utilitarian approach is such an interesting way to frame problems like this. It's obviously wrong for me to murder to my neighbor for their organs, even if those organs could save five other lives. But is it okay for a car company to roll out an AI program if it saves more lives than it costs? When viewed from afar the utilitarian mindset is always so alluring.

I think you're making a bit of a strawman here that people usually do with respect to utilitarianism. The utilitarian answer with 5 people on an island, where 1 is a doctor, 3 need organs, the remaining person has available organs, and the 3 are necessary to keep the group alive -- is to kill the one person and take their organs.

However, in a society where people can observe the actions of others and form motivations in response to policies, etc, you'll find that because society reacts fairly poorly to organ harvesting, because organ harvesting is implausible to do at scale without extra bad things happenning, etc, the utilitarian solution is actually not to go about doing it.

Only a naive utilitarian wouldn't try to also remain consistent with something like a Kantian imperative of global self-coherence.

Now, as for cars and testing self-driving on real folks, well, this may be something where the water is pretty murky. I think that society will react poorly enough to early bad events in self-driving that a measured approach is actually the best for saving lives in the long term.

The critique here shouldn't be that "well, utilitarianism sure looks good from afar, but would you murder your neighbor?" It should be "The problem is too difficult to address with utilitarianism because it involves complex societal factors and responses."

It's a bit contearian but if you will lose 100% of your population by not sacrificing 10%, then your scoiety is doomed in a way where normal ethics can't really apply. The guy refusing to give up his organs is going to be condemned as the naysayer nihilist who "wants us all to die".

I don't think replacing one set of risks with a smaller set of risks is at all morally similar to killing one person to save three.

A Kantian approach also works here though.

The person you're responding to isn't claiming that Tesla hasn't done a good job in the technology, rather in educating their users of its limits:

> Tesla has not done a good job educating their customers on the limits of their "Autopilot" system

The death was preventable, even with the technology deployed.

Where's the proof that it was a driver education problem? This argument gets rehashed every month or so here on HN.

Well, every time this discussion comes up, HN spends >100 posts splitting hairs as to what calling their system 'autopilot' implies.

Given that the technology is useless if you use it as 'intended' (Be aware, and in control of your vehicle 100% of the time[1]) I don't think this debate will be settled anytime soon.

[1] My 97 Avalon drives just fine when operated under those conditions.

Not to mention basing this on a single data point where a fatality occurred. How about the tens of thousands of other drives vs standard? That would at least be statistically relevant.

Even though what they released isn't a self-driving car but a glorified cruise control. I personally don't really see how that is reckless at all.

By describing and marketing driver assistance technology as self-driving technology.

Which is kind of funny, because every pilot knows the limitation of each kind of autopilot. I think a lot of it is public expectation of that kind of name.

Right, and if drivers read ALL of the fine print of Tesla Autopilot, they would have been informed of the limitations of that system as well.

The problem comes when the Large Print promises things that the system does not do.

Another problem came from the implementation of the system where Tesla did not verify that the driver maintains hands on the wheel.

The large print says to keep your hands on the steering wheel and be prepared for the autopilot to fail or disengage at any time. I don't see how this is promising a self-driving car?

It does now, but in previous versions it did not.

And even now, the popular perception is that Tesla is autonomous, and it is only the regulations holding them back.

googles cars have gotten in a few fender benders and side swipes

people have died using teslas autopilot

There was only one accident I'm aware of in which the Waymo vehicle was at fault; the vast majority have been humans doing stupid things like rear-ending them at stop lights and t-boning by running a red light.

(Disclaimer: I work for an Alphabet company.)

"At fault" is one relevant metric, but not the only one. An autonomous vehicle should at least meet, and ideally surpass, human-driven ones in avoiding accidents for which they're not at fault as well.

I'd be curious to know if Waymo vehicles experience more, less, or the same number of not-at-fault accidents per mile driven as human-driven ones.

That's a tricky one. Avoiding an accident for which you are not at fault could easily cause an accident for which you are at fault. Human drivers have a pretty bad record with that particular situation.

Also, there are some network effects at play here, the higher the percentage of self driving cars the more rare that situation should become.

I agree that there's a tricky calculus involved in some scenarios, but I think there's a much wider domain of not-at-fault scenarios where autonomous vehicles should be able to surpass humans unambiguously. Example: an autonomous car being tailgated by a human-driven car, with a stop sign or light up ahead. Different braking patterns have different likelihoods of causing an accident.

Autonomous cars really have to be able to handle these scenarios at least as good as, and ideally better than, humans: there's not a complicated split-second balancing act to worry about.

Tesla have stats saying their autopilot, for all its limitations, is already much safer than a human driver: https://electrek.co/2017/01/19/tesla-crash-rate-autopilot-nh...

Humans will do stupid things. A self-driving car has to drive safely in spite of them.

How do you avoid being rear ended at a stoplight?

There may be ways to avoid some of the other types of accidents, but those ones (which have happened several times) are difficult to escape.

The real question is what's the safety threshold - safer than median human? 90th percentile? 99th?

> "How do you avoid being rear ended at a stoplight?"

The same way defensive (human) drivers have been doing for a long time: leave room between you and the car in front of you. Gauge cars coming up behind you and roll forward if they need extra room to stop.

Does not prevent 100% of rear-endings, but a large portion of them. Most rear-enders are not cars plowing full-speed into you, it's someone misjudging their braking and not being fully at-rest when they needed to be. Leaving some margin for this kind of error helps avoid the whole incident.

Change lanes, allow the other driver to overtake, start braking a little earlier and more gently, touch the brakes to flash brakelights let the driver behind know that you are stopping soon. All ways i deal with tailgaters in different situations - it's a judgement call though.

Don't google cars drive on a limited number of preplanned routes with a very low speed ceiling?

That is more cautious, but that doesn't, on its own, suggest better tech. That is that the Google cars are actually safer, more advanced, etc.

> with a very low speed ceiling

Those are the Koala cars. Before those, there were the Lexus which went in highways too.

That also could be because autopilot is only usable on highways (IIRC) and the fatality rate per accident on highways is much higher than standard town/city driving.

Either way there's not nearly enough data to draw conclusions from.

Autopilot won't stop at stop signs or stop lights, so it is not as useful when not on highways.

Just to clarify the downvotes.

Highway driving is safer.

Google's self-driving cars started on highways (easier problem to tackle) before moving on to city streets.

There's a lot more tesla cars than google cars. And I don't think there was a single death directly because of autopilot (not 100% sure though). Also, autopilot is safer, so it actually _saved_ lives.

If downvoters don't believe autopilot is safer, here's article about NHTSA report: http://bgr.com/2017/01/19/tesla-autopilot-crash-safety-stati...

I'd guess the downvotes are more for this part:

> I don't think there was a single death directly because of autopilot (not 100% sure though).

There was a death, and it was very prominent.

Yes, I know about that white truck death. By 'directly' I meant situation when autopilot 'went crazy' because of some bug and sent your car in a spin on a highway etc. Probably should have used different wording.

I imagined as much. Even so, "autopilot sent your car in a spin" vs "autopilot ran into a truck" is a distinction without a difference for the people inside the car.

Tesla warned that driver still should be alert, they implemented warning so driver would keep hands on the wheel etc. NHTSA reported that driver had 7 seconds to react on the truck. In a spin situation because of some bug autopilot would create the problematic situation itself, even if driver will be paying attention to the road.

Yeah, the problem was letting people get the expectation that it was an autopilot when it wasn't. Decades of Human Factors research in aviation had already established that people can't maintain focus on a sufficiently-monotonous task.