I think we'd need a huge infrastructure upgrade to make life easier for autonomous cars. It'd have to be >100 Mb/s mobile data, with extremely good coverage (for the recovery mechanism you mention). Think GPS would need to be replaced too, the signal is way too weak and unreliable. We'd have to embed digital markers/emitters alongside lane markings, traffic lights, signs etc, so they're easier to deal with.
Connectivity you see is from a single mobile phone. A car could have a better antenna, service from all the mobile networks rather than just one, wifi fallback and a backup satellite uplink. If all those fail, you dispatch an engineer to find the car wherever it's stuck and add a solar powered LTE repeater.
If you have a map of the whole service area and know where you started and have a camera looking out the window the whole time, there is no need for GPS really.
Considering these are car companies doing this, true remote control is not a good idea. The industry response to the car hacks was pathetic. Only Tesla reacted properly. (And probably Google...)
Now you've reduced all your problems down to simply "identify problem and alert a remote human".
If each of those problems only happen once per day, and they are resolved in 30 seconds, it's going to cost only about $30 per car per year to have that remote expert on hand.