"Level 4 ("mind off"): As level 3, but no driver attention is ever required for safety, i.e. the driver may safely go to sleep or leave the driver's seat."
Level 4 is like a Level 5, but that doesn't let you enable it for every route (e.g. you might be manually driving until the highway, then enable self-driving and go to sleep, then wake up with it parked right after the exit you chosen).
Humans are able to "Level 5" driving, so...
...which is also partly the motivation to improve.
I expect self-driving cars long-term to be a lot like air travel. It'll have a reputation for being super unsafe, with lots of people paranoid of it, for decades after it has far eclipsed the safety of regular human car driving. The paranoid hyper-focus and ridiculous press coverage is also what will enable it to become essentially perfect (like passenger air travel is today in the developed world).
It's because in almoost all cases, one fatal mistake results in one fatal accident. And in most cases, the failing party is part of the accident - giving us reason enough to believe people to what they can to avoid it.
With AI, one sloppy mistake can be the root cause of 100 fatalities, with no risk to the one who made the mistake. That's part of the reason we don't accept it.
From a cynical perspective, it's easy to see a future where we constantly approach, but never quite reach, full Level 5 autonomy - or at least not with existing technology. Maybe Level 5 will prove to be an NP problem. Marketing teams are likely going to promote terms like "Level 4.1 self-driving" or "Level 4.5 self-driving" or "Level 4.9 self-driving", as edge-cases are slowly resolved.