Animats, you've been at this longer than anybody on HN from what I can see, question for you: What in your estimate is going to be the failure rate of these self-driving set-ups and what kind of failsafe measures to gracefully shut-down are still on the table once a major sensor fails?

I'm wondering about this because I miss this element in most of the discussions around the theme, a human having a heart attack or a stroke behind the wheel is almost certainly going to cause an accident (one sided at best, multiple vehicles at worst). But I don't quite see how that translates into sensor failure (or even CPU failure) in a self driving car.

Absent mandatory standards, that's going to vary with the manufacturer. Volvo says they have redundant computers, sensors, and actuators. They've probably tested fall-over to backup. Google has some sensor redundancy, but it's not clear how well they protect against component failure.

If I were designing this, I'd have three independent control systems. System A does regular automatic driving. System B monitors system A, but it's only as smart as a stock anti-collision system braking system is now. No AI. It has info from the radar and can slam on the brakes. Maybe steer a bit to get to the side of the road and stop. System C is purely a watchdog - if it detects that systems A and B have failed, it cuts propulsion and applies the brakes. If system C takes over, you may hit something, but won't run away out of control.

I'd ride in a Volvo or Google self-driving car. Uber, Otto, or Cruise, no.

So, something along the lines of the Shuttle computer design. That makes sense. Thank you for answering. I'm wondering about this because I expect the MTBF demon to kick in when the numbers go up. Right now the numbers are so small that equipment failure is negligible, but as soon as the numbers are high enough that these failures become more common the better architectures will start to stand out.

You will always have a redudancy of sensors in case they fail. The front looking camera usually does have a complimentary lidar and most cars are even equipped with both camera and radar in case a sensor goes AWOL. This will be the case is selfdriving cars as well. I think they will have even more redudancy then we see with ADAS cars today.

Every sentence you wrote contains the word 'will', so they are 'forward looking statements' and evidence free. I understand that we'd like this to be so but there is no guarantee that it will be the case and a single, non-redundant sensor going down could have immediate impact on the cars ability to get itself off the road in a safe way.

This wasn't about opinions but about the state-of-the-art and the facts as we know them today. What will happen in the future is anybody's guess, my guess is that before long there will be a couple of major crashes involving self driving cars resulting in mandatory redundant systems and redundant (quorum based, see Animats' comment) processing units. Otherwise self driving cars will end up with a bad reputation, even if the chances of failure with any individual vehicle are small.

Statistically speaking the number of components and the their internal complexity are good indicators of how frequently you can expect such a system to fail. Self driving cars are sufficiently complex that these are legitimate sources of worry and simply waving those worries away with what 'will' happen does nothing for me, it is the how that interests me.

Ofc they will fail at some point, there is nothing magical about a self-driving car. What I meant with the previous post is that it could be fail-operational see for example the volvo drive-me.

Whether having multiple sensors guarantees redundancy depends on what they are hooked up to, and how.

If the software they are connected to doesn't know what to do when the camera fails or starts sending bogus data or even to detect that it happens you still may have a lidar, radar, etc. but it doesn't make the system fail-safe.

Nope doesn't make it fail-safe at all and I don't think that is very hard to accomplish. What can be accomplished is fail-operational, if one sensors goes down using other sensors the car can navigate it-self to safety.