I am also worried about the adversarial nature.

1. People "hacking" cars for fun/malice. This could be anywhere from "painting on the road/sign" to wireless comms penetration attempts or nation state attacks.

2. People who OWN the cars replacing firmware/hardware for perceived benefits. IE put back the "rolling stop" feature. Or "improve" behaviour/performance. (Car modders changing headlights in a Tesla might mess with the optical only approach) Or change the default decision in "trolley problem" events.

It's even more interesting to consider the ethical engines built into self driving. If a car has no choice but to hit a man or a woman, which one should it choose? Adult or child? Two adults or one child? Should the car save a child pedestrian but risk the driver's life? This must be built into the driving self driving system or I don't see how it can be legally approved.

Okay, so you've got a private company explicitly writing down the value of human life in a big list, top to bottom. Let's pretend we fast forward the several years to decades for the public to accept this new reality. Now we realise that different locations operate under different value systems. In Saudi Arabia, one man is worth three women. You best believe they will require the car to run down the woman. So each country/location is going to require an ethics update, and it's all going to have to be public. Imagine the heat Tesla is going to take by complying with Saudi rules and explicitly programming their cars to mow down women instead of men.

I think that given that choice the car will choose "which is less likely to hurt my passenger" since the passenger is the (paying) client and not the pedestrian. Pedestrian infrastructure & defense will have to adapt to respond instead :).

I think this is the correct answer. It will be simpler and more straightforward from a legal perspective too, since it can be argued that any driver would do the same - they would try to protect themselves first, if for no other reason than the fact that humans simply don't have the ability to evaluate situations that quickly and play through a trolley problem in their head while the car is heading for a collision - you instinctively do whatever action feels right to protect yourself. So I think self driving systems will be coded with that imperative as well - protect occupants first and foremost, and they won't be calculating the "worth" of whatever or whoever they are about to hit. The extent of damage calculation will end at "there is one occupant in the driver seat, so the damage to them will be lower if I hit the obstacle with the passanger side of the car", but it won't actually know or care what the "obstacle" is.

Modern cars are already hackable - there has been several demonstrations where people connected to the car stereo, and through a chain of exploits could get access to the streering/breaking ICUs, essentially taking the car over.

Yet there has been no report of this exploit used maliciously in the wild.

> This is an important aspect of systems engineering that we haven’t even scratched the surface of when it comes to self driving cars.

There are definitely people in the industry who are thinking about this and working on it, but AFAIK none of them work at Tesla.

I'm assuming you mean commercial supersonic flight? I'm pretty sure there are plenty of military supersonic planes.

If you can do 95%, that might be good enough. Obviously if you could automate the entirety of the process, that would be ideal, but consumers may simply change their behavior to accommodate the 5% difference. Uber/Lyft already changed consumer behavior in a marked way: they allowed more people to move closer to the city, not pay for a car, not pay for a parking spot, etc. In Seattle before the pandemic, it was common for young professionals that didn't own a car to take Uber/Lyft most days. They did this so frequently that yuppies were blamed for the increased traffic, even though many of them were served by Seattle's great mass transit options.

The cost, simplicity, and availability of self-driving cars will undoubtedly change the consumer.

> If you can do 95%, that might be good enough.

Self-driving cars have been at 95% since the 90's. They've been at 98% since 2012. Humans are higher than 5 nines.

It turns out that human drivers, even poor ones, have a much better driving record than self-driving cars. With this latest Waymo experiment we may finally see some significant progress.