That's right now and that's why companies have to provide level 5 autonomy or there won't be any customers. Yet for days with good weather in urban environments level 5 might come within a decade; it's still better to be able to drive only half of the time instead of all the time. The long tail of problems that is supposedly plaguing self-driving systems exists with humans as well - how would you react if a bridge/building in front of you started collapsing or somebody started overtaking right in front of you?

I'd hit the brakes.

But then again, I am actually able to discern what is a bridge and what isn't, and thus don't have the need to suddenly cover my eyes and scream, "I'm confused and I don't want to drive anymore!"