Furthermore, even 99.99999% reliability is not good enough, and that's already incredibly hard to achieve. Five 9s of reliability still means 5 minutes of unreliable behaviour per year, which is unacceptable in the context of driving.

> Furthermore, even 99.99999% reliability is not good enough, and that's already incredibly hard to achieve. Five 9s of reliability still means 5 minutes of unreliable behaviour per year, which is unacceptable in the context of driving.

40,000 traffic accident deaths per year in the US.

< 400,000,000 drivers.

So more than 1 fatality per year per 10,000 drivers. There's a 99.99% change that you won't kill someone while driving in a given year.

That's just fatalities.

> In 2017, there were an estimated 2,775,608 police-reported motor vehicle crashes in which at least one passenger vehicle (i.e., a passenger car or a light truck) was towed from the crash scene in the United States, which resulted in an estimated 1,686,240 known passen-ger vehicle occupant injuries.[0]

So 0.6% per driver per year.

So far today I've seen one driver going the wrong way down a one-way road, and one run a red light. (Plenty entering the intersection on yellow and leaving on red, but at least one where it was blatant). I was on the road for maybe thirty minutes.

Do you think the average driver spends less than five minutes per year driving unsafely?

[0] https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...

Ok, point taken, I take my previous comment back. I guess we just have to wait until the risk posed by autonomous driving systems becomes socially acceptable. Right now it just seems like a scary unpredictable risk.

Just being pedantic, but those are seven 9s, not five. For five 9s, the downtime is indeed 5 minutes per year, for seven 9s it's about 3 seconds.

Still an interesting question: is 3 seconds of erroneous/unreliable behaviour per year acceptable from an autonomous vehicle?

I would be interested to meet a human driver who didn't have 3 seconds of erroneous/unreliable behaviour per year.

Depends what's happening. If those 3 seconds consist of randomly veering into something on a highway, that would be... Not good.

Even 5 minutes per year is totally acceptable. 5 minutes is 300,000 miliseconds. Split over the course of 365 days at 86400 seconds per day, we are looking at roughly 1 milisecond of erratic behaviour every 105 seconds. Not enough to crash a car

Perhaps that'd be OK, if the 5 minutes per year is evenly spread across the entire year (including time the vehicle isn't even being used?!)

OTOH, if it means that the vehicle is liable to have a 25-second "blackout" in the middle of your morning commute roughly once a month, it seems very different.

25 second blackouts are too much. However a 1 second blackout every 5 minutes might not be a problem in most cases: you can keep going the direction you are going and other cars that are not blacked out will avoid yours while the computer recovers. Of course this depends on the blackout - if the car is still steering itself but not looking for problems (a much easier problem) it works, if the car isn't steering itself it could run into a wall.

My practical experience with system failures is that they tend to not be evenly spread out in any kind of predictable pattern. Instead they tend to be acute and concentrated at a few points.

I agree that my assumption that failures will be evenly spread out is too optimistic. But on the other hand, if failures are concentrated on a few points, they should be that much easier to detect and correct.

You literally just proved theieindjri's point. (Assuming you're a smart person) these edge cases, even if they are clustered, even if they happen repeatedly, are NOT easy to fix. Partly because there are so many of them, partly because they only arise with a specific combination of events/circumstances that are difficult to replicate, partly because navigating them safely relies on facets of the human brain that AI doesn't perform, partly because the consequences of failure are so severe. "That much easier to detect and correct" vastly underestimates the difficulty of providing safe, automated driving.