The fully-autonomous route runs head-first into the inevitable, messy problems with software: security, maintenance, feature creep, complexity, unreliable networks etc. In theory, humans could nail all of these in car software. In practice, it's highly unlikely. The software industry can't even get basic security for IoT devices right.
The human-in-the-loop route runs directly into the only problem more daunting than reliable software: human nature. Human overrides for self-driving cars that need to be activated in a timely manner will not work. The drivers won't be paying enough attention to react in sufficient time. Note that overrides add even more complexity to an already fantastically complex system.
Say an always on self driving car, but it doesn't drive. Human has to drive, and the computer can correct any obvious mistakes the human does. Keep a really high liability for the car manufacturer though
ED: Not hitting kids is manditory, not blocking traffic is mandatory, parking in every random stop is optional as long as their are safe places it can drive to.
Basically, you don't need a perfect solution in every case, just a safe solution in every case. A self driving car that get's to a gas station is a good enough step 1 even if you have some human attendants there to fill up. Sure, it might add $2 bucks per fill up that's still cheaper than highway driving.
PS: Self driving car, only works in NYC is also useful.
For example getting out of the way of emergency vehicles on a blocked road. It will ask you what to do and if you tell it to drive up the pavement or into a bus lane or cross to the wrong side of the road -- the car will just do it.
I do wonder about all the non-verbal human-to-human communication. But a friendly baxter type screen with eyes that move (and messages that scroll across) would solve that.
We have to remember that these cars are all feeding unusual situations back to the google BRAIN that is learning. It'll watch us react to an emergency and learn. It'll see that 20 or 30 times a day and generalise a response.