You might say "your right to choose between houses is a privilege", but literally every culture in the world is going to observe differences between two houses, regardless of how they're allocated.
Even considering buildings where every unit has an identical layout, and all residents have access to the exact same amenities, one unit might be closer to the elevator and one unit might be closer to the garbage chute. These are fundamental differences between these units which make them not interchangeable.
> Endless rows of apartment buildings, each identical, supplying everything you could need.
"Providing everything you could need" is not how we define commodities. Fungibility is how we define commodities. One can argue that we should just be thankful with whatever home we get, but trying to act as if they're equivalent is just wrong.
Particularly if you're poor, location matters a lot.
The very rich will build a mansion in a beautiful but inaccessible area and commute to work by helicopter. The merely rich have the nice cars to get to work. But the poor have no car so need to be either very close or within public transport (mostly nonexistent in the US) to get to that job.
If you're without a home, one house is very much like another. It's only once you have money and ambition that you might want a better view, better schools.