A city-only car would be totally useful for people who live in cities; you just rent something when you want to go to the boonies.
Heck, most BEVs are that way now; I've got like 120 miles of range on mine, which is fine almost all the time. the two or three times a year I need something with more range or with more cargo capacity or what have you, I borrow or rent.
I think that the economics of the first level5 cars might be similar to current BEVs, in that you can only go where there is infrastructure. which is where most of us go most of the time.
But they still need to handle fault cases: e.g. they cannot only rely on a beacon sent by traffic lights because that beacon or the whole traffic light might out of order. So, imho, while smart infrastructure may help self-driving cars and traffic management it does not allow you to avoid the "hard work", which to make sure self-driving cars will behave safely and reasonably completely on their own.
Note, you still have the safety problems to solve. You need to know how to get out of the road if the road infrastructure is on the blink. I'm just saying that "I don't know what to do so I will pull over" as long as it doesn't happen too often, is an acceptable answer.
Also, a vast majority of the world doesn't even have proper roads like many western countries do.
In many parts of the world a road is not even paved or asphalted, and let's not forget what is actually making use of that road.
Seeing animals on roads might be a rare sight in a western country, but in much of the world, the road is shared by more then just passenger cars and trucks.
So this means that self-driving cars will have to safely handle these cases and that also means that it is likely that cars will have to still be able to be driven manually.
Bottom line: self-driving cars will have to handle absence of smart infrastructure (in which case do we really need that infrastructure? I think we'll still need it, though, to guide and improve traffic)) and/or cars will continue to be driven manually at least some of the time.