Mapping the world is hard because it is complex and evolve way faster than a Google Car army can patrol it for a reasonable cost. More often than not OpenStreetMap which is a community project is more up to date than commercial map, but it’s not precise enough. Beside nobody would bet there life on something that is Wikipedia-like editable by ill-intentioned trolls.
This is precisely why all autonomous car rely heavily on computer vision and why adversial attacks should be carefully considered.
Uncertainty will indeed « slam on brake » but adversial attack don’t aim to generate uncertainty the aim to generate false certitudes.
It's true these maps don't exist for the whole world yet but it's also why all sdc companies are launching in metros. They're building them.
Vision is good at recognition but bad at localization. This is the real problem. In robotics a fundamental problem is SLAM. You should read about it.
Adversarial attacks are concerning for any AI model, but autonomous vehicles aren't all AI. Lots of rules and heuristics and pre-knowledge are utilized. In the end if something is uncertain it generally amounts to "slam on the brakes" which is more a ux problem than a safety problem.