> when you do sell out your house has gone up in value a lot more than inflation, while if your neighbor had built a large apartment next door your house value wouldn't be as high.

This is an example of the issue I was talking about: it functions as an inequality accelerator. People who are already "in" have the power to set the rules of the game to be even more advantageous to themselves.