It's the other way around - you need a proof that they perform comparable to an average driver during expected use, and they do not have absurdly stupid corner cases during actually expected use.

Well, humans have absurdly stupid corner cases during actual use, too. It depends on how small the corner is, I think.

Yes and humans require certification before they are permitted to drive vehicles on public roadways.

To get that certification, do they test you first, or do they wait for you to screw up and then take it away from you??

We know, being humans, and that we have the ability to process lots of complex information in a way that's very difficult for computers to replicate. Hard AI doesn't actually exist (yet). We also have a 100 years of humans driving cars worldwide so we understand well what they're good at and what they're not, so laws & safety designs take all of this into consideration.

Each computer system will be encountering new, diverse things in the real world without a good understanding of how they'll perform. There are lots of crazy hard problems here that no one has solved yet. So to suggest we just automatically trust it because humans make mistakes is foolish when the consequences are so high. If someone came out with a surgery "autopilot" tomorrow, would you suggest it start giving triple bypass surgeries right away without FDA approval because humans make errors too?

One of the features of the common human firmware is self-preservation instinct. It lets us trust that our fellow drivers, while still prone to mistakes, won't generally make obviously suicidal errors. Can one say the same about a new ML algorithm running on some board designed half a decade ago? How exactly would one know, without a thorough audit?

They test you first, but if despite passing the test you screw up sufficiently badly later they decertify you.

It's a belt-and-suspenders system.

We've been dealing with those corner cases for thousands of years and we know them pretty well. Given that we all run on pretty much the same hardware and firmware (with minor differences), you could say humans have been thoroughly tested for few millennia, including a hundred or so years of road testing.

So yeah, I a bit of thorough testing of a completely new hardware running completely new software isn't too much to ask for.

When you measure the safety of autonomous cars by counting the miles they travel, and not counting the miles that they intentionally avoid or defer to a human, you have succeeded in creating a metric that looks like it's useful for comparison, but is actually completely meaningless. It's like measuring typing speed while ignoring typos.

Until autonomous vehicles are subject to the full spectrum of conditions that human drivers face, incidents/mile is not a meaningful measure of comparative safety. And until then, the burden of proof should be on the creators.

Not true. Lots of accidents happen in "good" conditions because humans suck at driving.

The way you compare, is you give a self driving car to a person, and compare actually usage for that person, and see if owning a self driving car increases or reduces accidents for that person.

Being perfect only half the time is still revolutionary.

The frequency with which an autonomous driving system relinquishes control is one important metric for estimating the actual risk. Measures like this are better and faster than using the actual tally of catastrophic events, because there is more, and more frequent, information. Engineers working in high-risk fields have been doing this sort of analysis for decades, and there is no reason to make an exception for autonomous road vehicles.

I have a 100% perfect safety record over probably hundreds of thousands of miles of driving a class B truck with my knees instead of my hands, using cruise control, on straight empty highways in Nevada, Idaho, and Utah, with perfect weather and good road conditions. There's nothing revolutionary about my ability to perfectly drive with my knees...I just very carefully selected the conditions where I was willing to do it.

You may have that perfect safety record, but other people don't.

People still crash every day in good conditions. That's thousands of lives that could be saved with our imperfect, sunny weather on the highway only self driving car.

You underestimate how bad human drivers are even in perfect conditions.

You are completely missing the point, so I'll try explaining via a Reductio ad absurdum hypothetical.

Lets say that Semi-Autonomous Cars are currently tested on about 80% of the tasks that humans currently face in the real world, and that through some feat of engineering and design we don't have to worry about the ridiculously messy transitions between Autonomous Mode and Human Mode.

And let's say that for those 80% of tasks, the Semi-Autonomous Cars have a 0.05% accidents/100k miles incident rate.

And let's also say that Humans on average have a 1% accidents/100k miles incident rate.

What you're telling me is that the Semi-Autonomous car has a much better safety record, so give us Semi-Autonomous RIGHT NOW OR WE'RE ALL GONNA DIE!!!

But what I'm telling you is that you don't know enough to make that decision yet, because you don't know exactly how well humans do on the 80% that Semi-Autonomous cars currently handle, you merely know the average accident rate over the current 100% of scenarios. For all you know, Human drivers could have their average 1% accident rate as a result of a 0.0% accident rate for that 80% subset and a 5% accident rate on the remaining 20% that Semi-Autonomous cars can't handle. And if that were the case, then forcing us all to use Semi-Autonomous cars would actually increase the average accident rate from 1% to 1.004%.

Until you fully understand what Semi-Autonomous Cars are capable of, AND know how well human drivers do on that restricted subset, you can't definitively say that current technology is better than humans.

I understand what you are saying, I am just disagree with the facts.

"Human drivers could have their average 1% accident rate as a result of a 0.0% accident rate for that 80% subset "

No they couldn't, because they don't.

I am asserting that for this specific 80% of perfect conditions, humans are still terrible drivers. And that being better than them is EASY, because of just how terrible humans are at driving (even in "perfect conditions").

> No they couldn't, because they don't.

The numbers were deliberately exaggerated to make the point. The fact stands though that until you know what the numbers are, the best answer is not easy to come by.

> I am asserting that for this specific 80% of perfect conditions, humans are still terrible drivers. And that being better than them is EASY, because of just how terrible humans are at driving (even in "perfect conditions").

And I am asserting the opposite: that the appearance of safety of autonomous vehicles is the result of highly selective conditions with near laboratory levels of control, the likes of which are so monumentally easy to handle that even humans, as shitty and inattentive as they are at driving, can handle with comparative levels of safety. And I certainly think it's possible that computers will lag behind humans for another 20-50 years while we slowly develop the massive body of fast-heuristics research necessary to make NP-complete planning decisions with the speed and capability of even below-average humans.

>>humans suck at driving

Proof?

Well, roughly every driving week or so...

Just two days ago, driving on a roundabout some lady enters the roundabout right in front of my car. Fortunately it was a two lane roundabout and the left lane was empty so I could swerve to avoid crashing into her left hand side door (I was coming straight at her).

At the traffic light I asked her if she had not seen me and she went 'Seen what? Where?', so apparently she had not seen me at all, which is pretty impressive given that I was less than 5 meters away from her when it happened.

This sort of thing happens with some regularity. All it takes for a situation like that to turn into an accident with possibly a fatality is for me also not to pay attention.

so we should wait till a statistically-significant percentage of people die before we tell everyone, "yep, definitely dangerous. Lets add some regulation" ?

I wonder if they could shut him down on the basis of car insurance requirements and the legal definition of "driver". If "driver" is defined as the "person" inputting control commands, then either you dont consider the computer a person, and then nobody is legally "driving" the car, or else the software somehow meets the definition of a "person" or driver, in which case its an unlicensed driver operating the vehicle and insurance may not have to cover it.

The NHTSA has been a strong proponent of self-driving technology and has gone out of its way to make sure there is a path to doing it legally, without pulling the legal 'gotcha' that you described here.

The device would probably be SAE Level 2, and should be regulated as such.

beyond the obviousness of what you have overlooked, its also worth pointing out that not all autonomous car usage will be substitutive. What if i just send my autonomous car to go take a picture of something? I might not have done it before, but now I will because it only costs a few cents/mile and none of my time.

Another really cool use case will be sending your car to drive around the block for an hour or go back to your house and park, while you are going out to places. Beats paying the $10 to park for an hour.

Lots of interesting effects there, all the way up to basic urban land-use decisions.

Suddenly the price people will pay for parking is going to be closely related to the price of gas. The maximum value of a parking space in the city is basically the cost to send the car back out to some suburban parking garage during the day and have it come back in later for you. My guess is that the market-clearing price is going to be a lot lower than it is today. Lots of urban parking structures might end up getting repurposed for other uses, if you can't fill them at hundreds of dollars per space per month.

In the short run I could see the average self-driving car's gas usage being higher than a conventional car's, but in the longer run the same self-driving features might make fully electric cars more palatable. Long charging times don't really matter if you can let the car go and charge itself while you're at work, for e.g.

And if that place that it goes to charge itself happens to have a pile of solar and wind, even better! Way easier to do a bit installation in a less populated area than right in the centre of downtown.