> we all still get in cars believing we'll be better than other drivers. Who's going to get in a car when any given car has an equivalent chance of making a bad decision all on its own?
Given the fact that plenty of people take taxis, rely on Uber and Lyft drivers they've never met, and take public transportation like aircraft and trains with little knowledge of the operator, this seems like a non-issue to me.
So if they eliminate all of the accidents that are caused that way, but increase the number of garden-gnome related accidents...wouldn't that be worthwhile?
Another, less flippant way to say it: yes, we can imagine scenarios in which the number of accidents will increase. But what is the relative frequency of those types of accidents compared to run-of-the-mill rear-end, red-light run, lane change due to inattentiveness?
There's also the issue of what a self-driving car could do... and what they WILL do. Much technology COULD be amazing... if companies did not function in the ways that they actually do. Because of that, technology often fails to meet any of the promises made. Development is supposed to be cheap and fast, and safety and security get left at the door. No company is held liable for such things, and it would be expensive and bothersome to fix, so they're not about to bother.
Humans may or may not be less situationally aware overall, but we have the gnome question down cold. Cars don't. They see "obstacle in front of bumper radar."
The OPs point is well-taken because it illustrates a technical risk that few are discussing: the "last 1%" of self-driving software may be equivalent to solving AGI.
I don't see why the "last 1%" problem needs to be solved when we have hundreds of thousands of accidents every year caused by people not paying attention, not reacting in a timely fashion, and driving unsafely; all problems that computers are likely to excel at.
Exactly right. You need a semantic world model, reasoning and prediction. AGI bottlenecks self-driving, assistants, text understanding…
Superhuman sensors will be great, but you really can't dodge AGI for true full self-driving.
Considering humans can barely stop when they should, I don't give them a high likelihood to also be able to have the situational awareness and make the multiple split second decisions required to both decide to, and take action upon, directing their car towards the least-valuable obstacle in an emergency. By the time they decide what to do they will probably be rear ended.
Edit: your examples also ignores that with self driving cars, people won't be put in these situations in the first place. A driver not paying attention to the road, looking at their phone would be... not a safety hazard.
The OP's scenario is contrived to make a clear example, but it happens regularly when you're driving: you're forced to make a choice between acting and possibly hitting something, or doing nothing and being hit.
We don't need self driving cars to solve the 80% case of preventing human drivers from hitting things. We need radar and auto-braking, which already exists. Full autonomy requires solving the harder problems.
[0]https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
If all the cars were autonomous, it would more likely to be a matter of frustration than safety: the gnome causes a traffic jam because the cars won't endanger the passenger by driving into the opposing lane. Then what? They find an alternate route? They automatically call in a service crew to clear the road? Not the end of the world.
I also think people incorrectly think that the moral choice dilemmas that self-driving cars will face are somehow the difficult part. What will happen is that we will provide valuation functions that are somewhat close to our intuitive collective understanding of value of life in different forms. Computers use them to evaluate these situations and we as a society just adapt to the new ethics of the AI age.
I also want to see the CEO, CTO, and other C-level execs of the auto company to release a legally binding agreement holding themselves personally liable for criminal negligence if the vehicle is later found to have obvious flaws - obvious determined by an independent group of several thousand unaffiliated volunteer experienced software engineers.
Right now, if a car company slapped a 'self driving' car together that was designed to seek and destroy pre-schoolers, no one would say boo. Certainly not the courts. Toyota killed people by having ludicrously egregiously bad development practices (their developers did not even have access to a bug tracker... they followed 6 of 94 industry standard recommended practices... and worse) - and the courts acquitted them of negligence. That is disgusting, and makes it impossible for any reasonable person to trust any auto company doing development of something as significant and difficult as making a self-driving car.
They know that they will not be held liable no matter how bad of a job they do - would you drive across a bridge built by bridge engineers hired, managed, and made to work the way software engineers are hired, managed, and made to work in modern companies?
Here's what I mean. Imagine this situation:
You're pulling into your driveway in your self-driving car, and one of your kids left a lawn gnome there, blocking it. The car won't go forward because there's an obstacle.
Simultaneously, a driver looking at their phone and not paying attention comes careening down the road. They're clearly going to rear-end you, hard. You COULD just pull forward and out of the way.....but the car won't, because of the lawn gnome. It can't know the lawn gnome doesn't matter in comparison to the damage your car is about to suffer.
Imagine this flip scenario:
You're pulling into your driveway in your self-driving car, and one of your kids, a toddler, is in your way. The car won't go forward because there's an obstacle. Otherwise, same situation as above. Now, you ABSOLUTELY do NOT want to pull forward, because you'd rather get rear-ended than run over your child.
Even if you want to nitpick this exact scenario, surely you can think of a hundred similar contexts yourself, where driving involves understanding the value of different pieces of the car's immediate environment. A car cannot know the value of a child versus a lawn gnome, nor can it reliably detect the difference.
Even if such a situation only arises once in a thousand days of driving, with hundreds of millions of people on the road everyday, that's thousands of times per day. That's really bad.
Is it statistically better than the amount of harm done by human drivers? Possibly. But we all still get in cars believing we'll be better than other drivers. Who's going to get in a car when any given car has an equivalent chance of making a bad decision all on its own? It's just like getting in a car with a drunk driver.
Who's going to sign up for that? Not me, that's for sure.