Having a human on call to remote into a Car is still a viable option in most of the US. This is not a good fit for sub second response times or tunnels, but it vastly reduces the endless .0001% edge cases.

ED: Not hitting kids is manditory, not blocking traffic is mandatory, parking in every random stop is optional as long as their are safe places it can drive to.

Yes, it may provide a good impetus to upgrade our mobile data infrastructure. It would have to be ultra fast, ultra reliable, ultra wide coverage. Because if you can't remote in, and the cars caused a huge traffic jam, it'll be very difficult to get the owner companies human in to help out the vehicle. I guess the recovery man can just use a bike :/

It still needs to work 99.99% of the time. I am more thinking extreme edge cases like: "Drove off the highway to avoid collision and stuck in a ditch. Bridge out now what? etc."

Basically, you don't need a perfect solution in every case, just a safe solution in every case. A self driving car that get's to a gas station is a good enough step 1 even if you have some human attendants there to fill up. Sure, it might add $2 bucks per fill up that's still cheaper than highway driving.

PS: Self driving car, only works in NYC is also useful.

And if you can remote in, I'm worried about hacks. Imagine the chaos if a terrorist group got in and turned the wheel of an entire fleet in a random direction. It seems to me anything can get hacked/any individual can be pressed, and it's alsways just a matter of if it's worth the trouble. Seems a hack like this is would be worth some, to some groups. Also: they only need to succeed Once. The rest of us need to have perfect security among all fleets of all corporations and brands for all time.

If you can call for a driverless Taxii then there is some level of remote access. So, it's going to be a concern in either case. However, you can set the car to dial out when it's confused rather than waiting for remote connections which reduces the attack surface.