Rather than a fallback driver having to physically enter the car, maybe they could take the drone pilot approach and control it remotely. There would be the latency to overcome, but the built in crash avoidance might be good enough for a relatively short/slow distance to exit the exceptional situation.
That's actually a really interesting idea - hand off driving to a remote person when the conditions are bad.
But would a car's connectivity and latency allow for that to be safe?
Edit: If not there would still be some form control needed for the passenger.
Uber could probably also just have defensive systems that simply park the car and wait for a human driver to arrive, if there is a problem. It's good business for them to have 3 cars per driver. It still doesn't approach the interesting problem though, which is autonomous driving on a human level (Autonomous Uber without backup drivers, in an entire city). That I think is far away.