Perhaps you should spell out this easy argument. On the surface it makes no sense to me.

Bob down the street having a mansion doesn't impinge on my rights in any way. In fact, it is great that Bob's mansion doesn't get taken away for my sake, because it means I have the opportunity to also get a mansion that won't be taken away from me :)

If you look for the problem where it is non-exsitent the thought will surely make no sense, but instead of a mansion in the suburbs, think of a highly populated area. Think silicon valley for instance, where 150k does not allow for a decent living condition. Take my home country, Brazil: 6.9 million homeless, 6 million empty homes (https://www.bbc.com/portuguese/brasil-44028774). The exact nature of this we can only speculate, but given how crazy expensive flats in big cities like sao paulo cost and how many of those are uninhabited, we can only blame speculation and the market regulation for it.

That is some of the thought behind my argument.

I would disagree that everyone needs to live in sao paulo or silicon valley.

I never said that. I am just describing a problem. Those who live there do so because they either have jobs, family or some other need. In any case, that would not be a problem if, like in Japan's most populated zones, there wasn't this disparity between land pricing and uninhabited zones.

That is however a poor reflection of reality. In reality every man is not living on some Robinson Crusoe island where they create all their success and wealth in a vacuum.

Quite the contrary, almost all wealth in modern society is dependent on utilizing the knowledge and labor of other people. Either you extract rent from owning property or you extract profits by owning capital.

Either way you are somehow getting other people to do work for you. As your wealth and success grows your bargaining position improves.

In a market economy, the price paid for a product is determined by the bargaining position of the seller and buyer. A sufficiently rich capitalist owner can squeeze down the wage of his workers and thus extract a large surplus of income.

With wealth comes power. You can use your wealth to buy or influence politicians to create laws that benefit your and which are detrimental to your workers or possible competition. You can use your wealth to propagate a dynasty. Pay for your children to get the best education. Send them to schools where the rich and the powerful create special links and bonds. An insider game where they make sure they stay on top.

It is native to assume the game cannot be rigged in favor of the rich. It can and it has been done so since the dawn of time.

It does not mean we should not have rich people. But let us not kid ourselves into thinking opposing and conflicting interests don't exist.