If you want to create a fair system you should also build infrastructure otherwise it is unfair when someone lives near the subway in a historical part of the city and someone has to live near the forest among ugly modern high-rise buildings.

That is how things are done all over Europe though. The state plans a new suburb by also planning bus stops, roads, subways, trains, parks, shopping malls and schools within walking distance etc. That ensures there are no really bad homes so even the cheap homes are in nice livable areas.

It isn't expensive to build such areas, so there is no reason not to do it except if you want to punish the poor.

Almost by definition the cheapest homes will be the “worst” homes so the goal is to make them as less bad as you can.