> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say.

And people would not drive into a river passing through multiple barriers, just because their GPS says so.

https://theweek.com/articles/464674/8-drivers-who-blindly-fo...

https://indianexpress.com/article/trending/bizarre/driver-in...

A few people taking completely stupid decisions is always going to happen. There's also drunk people or elderly people with poor vision who end up in the wrong direction on the highway. Even if it happens every day on a global scale, it's an incredibly small fraction of drivers. A targeted attack fooling current-technology IA could lead hundreds of cars in the wrong direction at the same time in a single place. It's way worse than what you could imagine tricking humans to do. Putting such brittle systems in charge on the road is irresponsible.

Hundreds you say? Think again, it's happened. (Well, 100)

https://www.google.com/amp/s/www.cbc.ca/amp/1.5192656

That's a fun trivia, but that's not really relevant to the discussion here: those people took no risk for their lives when following these instructions. Making hundreds of people go in the wrong place is easy, but the difference between a bad IA and a human driver is that the majority of drivers realize when they're doing something dangerous and stop.

This common sense of danger is what IAs must have before we can trust them on the streets.

For instance in the Uber fatal crash: the IA correctly labelled the cyclist as a cyclist several times then revised its judgment, then definetely labelled it as a cyclist but “oops too late”. A human would be like “wait, I think I saw a cyclist, slow down and figure out what this really was”. Obviously with the current computer vision technology you cannot do that, otherwise your car would be afraid of everything and refuse to even start, because there's way too much mislabelling happening at some point or another.

I just meant it as an example where a large group of humans will follow what a computer tells them even when it is pretty clearly wrong. It's not hard to conceive of possibilities where the danger wouldn't be obvious until after person blindly follows the instructions.

> A human would be like “wait, I think I saw a cyclist, slow down and figure out what this really was”.

This explains why there are only 5.25 million car accidents a year in the US.

Most of them being benign collisions though. “Only” 34k of them are fatal. For like 270M vehicle in circulation.

I don't know how many automated vehicle Uber has, but with one killed already the ratio is a few orders of magnitude worse for their automated vehicle than for the average driver in the US. (probably even way worse than the average drunk driver actually)

> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say.

You might want to watch the one-way roads in big cities. It happens a lot more often than you assume.

It also is (usually) self-correcting: oncoming traffic will honk, stop, or move around. The offender will (usually) realize their mistake and try to correct.

Sometimes, though, that's not enough. Searching "killed in wrong way one way" on DDG (or assumably Google) yields many (!) news stories.

Been there, done that (except the "killed" part). It was in a heavy fog. I was doing well to find a street at all, and it turned out to be one way the wrong way (the only such street in town). I figured it out when I saw wall-to-wall headlights coming at me out of the fog, and made a fast move for the curb...

So, yeah. People react. Which brings up the question: How well do self-driving AIs respond to a wrong-way driver? How well do self-driving AIs recover when they are the wrong-way driver, and they suddenly have enough data to realize that?

I'd suggest a slight correction here: You don't need the qualifier of 'big cities'. I live in a big town/small city it happens all the time here too. And when I lived in a small town that had a one way drive around the square, it happened a lot there too.

It's so frequent in fact that the barflies at one local place (that has a beer garden from which you can see a one way road) turned it into a drinking game - wrong way car == take a shot.

You put too much faith in humans. Things like stop sign removal have caused deaths in the past. https://www.nytimes.com/1997/06/21/us/3-are-sentenced-to-15-...

Humans are not so bad as drivers. Your example is an event from over 2 decades ago and was deemed newsworthy. Humans drive in all kinds of conditions but death rate is about 1 per 100 million miles driven. A search reveals crashes to be on the order of hundreds of collisions per 100 million miles driven. Age, country, intoxication level, road design and laws, road and environmental conditions also play a major role such that accident rates for someone aged 30+ in a Northern European country are going to be a lot less than teenagers in a country where road laws are merely friendly suggestions (and considering the chaos of driving in those countries, the rates are actually surprisingly low).

I will go further and say that almost everytime there is an accident the driver is somehow impaired. Lack of sleep, drugs, illness (old age included, mental disease), poor judgement (young age included, emotional distress)

Humans are surprisingly good at driving under normal conditions.

Some are. Some are not. Last week, I was nearly in two accidents on maybe a 1 mile trip to the store from my house. Both times were people pulling out of traffic, ignoring right of way. I prevented the accidents that would have resulted from these two separate idiots. I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault.

In my experience, I've not been in an accident with a teen, nor someone elderly, though I know people that have (both causing and being involved). Neither have I been in an accident with someone that I could tell was impaired by drugs or alcohol. I don't know for sure any of them involved a phone for that matter. Weather was only a factor in one accident (pouring rain, low visibility).

I have nothing to suggest that any of my accidents were caused by anything other than inattentiveness, even the one time weather played a minor role. I also see a lot of dangerous behavior every time I drive: people running lights and stop signs, completely ignoring yield signs (seriously, they must be invisible to everyone else), failing to yield right of way, failing to signal turns and lane changes (my favorite is turning the signal on after moving into the turn lane), lots of phone usage (for everything except making a call, from maps to texting to watching videos!).

> I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault.

Do you just drive a lot or do you brake too late / too hard? Because an accident rate that high is rather unusual.

No, I don't drive that much. Most of my driving is in bumper to bumper traffic on 3-4 lane roads, though and people do tail gate a lot. None of the accidents have been because I've braked too hard. Of the most recent accidents, two were due to people not checking their blind spots while changing lanes. One, I was waiting at a stop light and the car behind rolled into me. And the fourth, a neighbor backed into me. Most of the accidents I've been in are after having been stopped at lights for nearly the entire duration before the accident occurs. The worst, I got hit by a guy speeding in a 45 mph zone while stopped at a light. He totaled his car. I've also had several hit and runs while I wasn't even in the car.

As I stated in my post, most of them are due to people being inattentive and not following the basics like keeping a reasonable distance checking blind spots. Others I suspect phone use, but can't prove it.

Yeah, the “I'm like 5 sigmas deep in the probability distribution but I bear no responsibility” sounds a bit suspicious.

> or do you brake too late / too hard

You mean live in a place where drivers tailgate?

> I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault.

That's way too many to take your word for it. Where there's smoke, there's fire. Well, maybe not... but you sure as shit should suspect a fire.

> a country where road laws are merely friendly suggestions (and considering the chaos of driving in those countries, the rates are actually surprisingly low)

I wonder how well self driving algorithms would compare if tested in such a "hostile" environment? Perhaps we shouldn't allow them on more "friendly" roads until they can consistently surpass human performance under such adverse conditions.

the claim is not that automated driving systems are ready for use, the claim is that if you do things in order to compromise a system that has a good chance of killing people and then does kill people that should be illegal, which of course it already is.

Yeah. "Voluntary manslaughter" and "malicious mischief" are already things you can prosecute for.

You could spray handful of nails in the road and I think there is a big chance it would cause an accident. Or you could just dig up a hole using tools available in most homes. Agreed, it's not that easy, but not hard as well.

Yeah i think if you start digging in the middle of a busy junction, people will have some questions for you

You would think people would have questions for bike thieves using angle grinders but nope. Just throw on a high-vis yellow jacket.