Getting to level 5 will require "V2X" technology - vehicles communicating with other vehicles and the road/traffic infrastructure itself.
We will need a forward-thinking government to start building sensors and communication tech into roads, stop lights, parking spots, and a secure/interoperable internet for all these sensors to communicate autonomously.
Currently the cost of implementing such a system is way higher than the return to drivers who can...what, work on more powerpoints or facetime while they're in the car instead of driving? Not to mention the huge security and safety and liability risks this would open up.
That's why we solved this problem with public transportation in the past - hire one person to be the "V2X sensor" AKA bus driver.
“Never” is kind of a strong word in this context... My personal belief is that we could very well have level five autonomy within the decade, using only two video cameras and two microphones behind a windscreen. But this would of course require major progress in A.I., of the sort that may also not happen in a hundred years.