Eminent domain is one establishment-sanctioned way in the US, wherein the owner is forced to sell to the government. The purpose is supposedly to enable construction of public infrastructure such that private owners cannot obstruct for personal profit, but the controversial Kelo v. New London case involved the use of eminent domain to transfer property from one private party to another.

If your style of ownership is a clear burden on the surrounding community, they will eventually strip you of your property. But the bar is set rather low, even for the minorities in a racist community. As long as you pay your property taxes, obey the most literal interpretation of the law, and don't allow the property to look too bad from the curb, people will generally leave you alone.

So the duty of the landlord is to not be such an enormous ass that the community notices it, and comes after them. It isn't difficult, let some people can't even manage that.