Frankly, that sounds like a motivated reasoning. It just seems that everything is about property rights and market, until it is inconvenient in which case tax is set up so that people who are not you leave, because you want their place.
And you still did not made place for more people. Just changed who is living there. They dont matter, you do.
This would be the ethically fair system. Of course in the present world the landowner class successfully lobbied that the fees are subsidized by other people from their income and similar taxes. There you have an explanation of the perpetual housing boom, because it is highly subsidized investment and there is only so much of it.
This doesn't apply just for land but for anything life necessary which can't be easily created anew. If someone poisoned 90% of water on earth you'd have the same thing.