In the hills of LA you have sharp blind corners where people have installed public fisheye mirrors to help you see around, then you have crazy people in Hollywood throwing furniture in front of your car, and non-stop traffic and people passing on the wrong side of the road between blocks even when there is a median, school kids and crossing guards, emergency vehicles trying to through and people doing otherwise illegal things to help get out of the way…
Anyways we’ve gone from “this won’t happen in our lifetime!” to “it doesn’t handle X niche use case yet.”
In a city (especially in SoCal and the American Southwest, which is, AIUI, where all the self-driving cars are today), you can be nearly certain that the various mapping companies have accurately plotted the roads and destinations, and if you're trying to get to a popular Finger Lakes winery, you won't be directed down a limited-use seasonal road that's entirely covered in ice.
In a city, you can be pretty well guaranteed that there are speed limit signs anywhere the speed limit actually changes.
Just off the top of my head, as someone who's lived 40 years in the rural Northeast.
They may be where you are, but they aren't generally here.