There's a straightforward solution to this that doesn't compromise public safety:
Employ paid training drivers who monitor the AV while it is collecting training data in environments where it does not yet meet the safety bar. These drivers are paid to be observant and intervene to avoid accidents. Furthermore, in their interventions, these training drivers provide critical training data used to improve the system.
This is what Waymo does also.
Perhaps Uber should have been more careful whom they hire, or put stronger controls on what they are allowed to do in the car.
Nobody wants to let a minimum viable driver chauffeur them around every day. Accepting the reduced quality of performance and increased risk a short duration cab ride is another story.
That's a good thing, we shouldn't be lowering the bar for AVs.