I expect a lot more from AI. Years ago I could play Quake III Arena and get pwned by bots.

I expect modern AI to recognize every car, pedestrian and physical object. I expect it to recognize all hostile targets and avoid them when feasible. I expect it to predict every possible move for both automated and manually controlled vessels and pedestrians. My AI needs to have an anti-gang mode that will protect the occupants even if that means acting as a weapon against hostiles. After all, people may be sending kids to grandma alone in the car (you know they will).

I expect all of this to occur without the need of network connectivity. My vehicle is to be prohibited from communicating without my express permission until after reviewing the data it wishes to upload.

I would like to add, if the vehicles entirely depend on their AI, then there should be at least 3 or 4 compute nodes and a voting system. Should a node fail or have false information, the other nodes should override.

I think, I agree that we should expect more from AI drivers. But how much more?

Even if they managed to decimate the number of fatal accidents, I'm sure that a large portion of people would still be really irrational about it and that worries me a lot

> I'm sure that a large portion of people would still be really irrational about it and that worries me a lot

I used to think that. The Tesla AutoPilot accidents have inspired no such reaction, though. The moment might come more when a self-driving car is used by a terrorist or school shooter as a getaway vehicle than because of someone getting mowed down by an errant algorithm, the latter being a cause of death we've been relatively numbed to.

Even if the cumulative amounts of fatal accidents is reduced, certain types of accidents happen less, if different types of accidents happen more often, say accelerating into a wall uncontrollably, that is not optimal. I worry that we will accept 'slightly better' as good enough, along with the consequences that come with it.

At some point, economics and actuaries come into play. It will be worrying when 'AI is safer than humans!' can be pointed to as a reason to not correct expensive, perhaps intractable, AI failures.

That’s all in a hypothetical future in which autonomous vehicles work as intended, are well tested, and are better than people. We’re still nowhere near that kind of lvl 5 automation, and just testing on open roads.

Be careful of the utopia trap: “anything is acceptable now for the rosy future imagined.”

In a few years, I think this is a more likely concern:

"If my child died in a situation where I know without a doubt an AI would have avoided the accident, it would matter a hell of a lot to me."

Absolutely, you're right. But as a singular entity, I'm a human, and an AI is "other". I know I perform better in 2 cases, and AI statistically perform better in 100 others, but I believe I'm more special than statistics in those 100 other cases, so I only care about the cases where I could have made a saving difference.

I guess I'm suggesting there's bound to be a (n irrational) paradox where everyone wants other drivers to be replaced by AI, so they themselves can drive safely and avoid the edge case accidents.

I’m looking forward to having a self driving car. You value control so much that having amortized years of your life back isn’t worth it? How many hours of your life have you and will you spend driving when you could have been surfing the web, talking to loved ones, reading, eating or literally anything else?

The problem is not self driving cars but the unsafe driving cars, you know the move fast and brake things, self driving AI is not ready yet, when it will be ready and safe I will accept them, I wish we get safety at the level that NASA or airplanes not the safety we have in web apps and regular software.

...but near term "If my child died in a situation where I know without a doubt an AI caused the accident, it would matter a hell of a lot to me."

There are going to be a lot of lawsuits that will sort out the legal frameworks after the first wave of failures by automated conveyances....

It could easily be the other way around. Right now cars are murder machines. Your child could be hit and killed by a human driver where an AI would have avoided the accident.

Or a self driving bus with children could get confused by a weird sticker and jump of a bridge, hit a giant truck stopped in the middle of the road.

We are speculating here, we need numbers to see if this self driving cars are better, how do we get this numbers though? Is it fair for the citizens of a city to be forced to be part of this tests? Are we sure the correct number of incidents are reported from this tests? are the tests good enough for real world (are they testing in all conditions are avoid some streets or some weather to keep the numbers looking good)

They're testing in California. I.e. flat-ish, wide roads, relatively nice weather. Sure, it's ideal for baby steps, but passing this off as "look at our safety record, therefore it's safe anywhere" is disingenuous at the very least.