Things are way worse for all kinds of changes to roads. Even if the first car correctly classifies that white spot on the road as a lost paper that it can drive over, what good does that do the next car? The wind may have blown it away or to a different location and into a different shape.
To me, Google's approach seems an attempt to build a model of what the entire world looked like a short while ago, while cars only need a rough model of what it looks like now.
Scaling Google's approach to millions of cameras in million of cars may improve the model and decrease its latency and might make the latency low enough, but I don't see why it would be the best approach.
There are a tremendous number of things to map. If they could all be understood automatically google wouldn't need to map them ahead of time.
If the car was able to do that simply by slowing down you wouldn't need the map in the first place.
The whole point is that a human needs to map it. The car has no idea what to do.
For example it doesn't "see" a stop sign and act on it. It knows in advance there is a stop sign there because a human told it so.
It's not scanning the environment looking for traffic signs, all it's doing is looking for obstacles in the way and avoiding them.
It doesn't even see the road edge, or the lane markings - it knows that in advance.