I agree and, yet, you need something at least very close to that before you can have cars without anybody in them shuttling around to pick up people or deliver a pizza. (Which is why anyone who thinks they're going to be able to hail a roboUber in Manhattan within a decade is... optimistic.)
I suppose you can construct really elaborate corner cases that you might give a car a pass on. And maybe there are ways to make things like construction zones easier for cars to navigate. But, yeah, for the most part, the computer is going to have to handle any driving situation a typical human would be able to.
It'll be a while before the human is completely out of the loop, but Waymo has filed a patent for 'remote assistance in low confidence situations', and many of the companies pursuing robotaxis have signalled similar initiatives.
The handoff speed is an issue as is the fact that the vehicle would have to recognize a problem in order to ask for help. But, yes, some sort of remote assistance system would seem to be one solution to at least certain edge cases.
There's no point in discussing L5 realistically, nobody is anywhere close to pulling it off.