Ok. But this idealism doesn’t work when Tesla has clear failure modes that simply shouldn’t happen during normal driving.

Humans generally do not give their cars haircuts by slamming them under a stopped cargo truck. They do not generally smash straight into stationary emergency vehicles.

We can’t handwave basic safety issues away by saying “in aggregate, they perform better than humans in most conditions.” The basic safety issues get people killed. Leaning on some “average driver” fallacy is a way to ignore issues core to the tech stack.

Slight quibble: I very much agree with your overall point but human-driven cars slam straight into rescue vehicles all the time. They're often intoxicated but not always. There's just something about flashing red lights that attracts driver attention and all too often the car follows.

This is why we put the BRT (big red truck) behind the rescue scene and park it at a 45 degree angle. It weighs 15x as much as a car so it won't move much when a car hits it, and we want the car to bounce off sideways and away from the rescuers (us) when it happens.

Assuming that in aggregate they do kill fewer humans, then this is a rare instance of an actual trolley problem. Do you want to kill fewer people by changing the norms and reasons?

Big assumption to make if it turns out the numbers and data don't scale up once you do large-scale deployment!

You're not wrong. This should be done with great care.

why doesn't it work? isn't all our society always abstracting away the individual stories behind statistics? (crime is down by 10% ! GDP is up by 0.2%, birthrate is down by xx).

we're used to it and we're happy about it and we use it to make decisions (buy house now, move to a new city ...).

Plus in those rare cases that you mentioned at least the same system will be feedback the incident to make sure that this won't happen again in the future... so in a morbid way there is a way to learn from such fatal incidents ... you can't really say that about such incident if the driver was a human.

It’s not a fallacy, it’s a risk analysis that will save lives.

This seems to be a reasonable take to me. Systems like Autopilot make driving safer. I'm worried that the conventional wisdom on HN is (a) that systems like Autopilotmust be perfect, which will never be attained, and (b) extremely loathe to recognize the times Autopilot has saved lives.