Eminent domain is one establishment-sanctioned way in the US, wherein the owner is forced to sell to the government. The purpose is supposedly to enable construction of public infrastructure such that private owners cannot obstruct for personal profit, but the controversial Kelo v. New London case involved the use of eminent domain to transfer property from one private party to another.
If your style of ownership is a clear burden on the surrounding community, they will eventually strip you of your property. But the bar is set rather low, even for the minorities in a racist community. As long as you pay your property taxes, obey the most literal interpretation of the law, and don't allow the property to look too bad from the curb, people will generally leave you alone.
So the duty of the landlord is to not be such an enormous ass that the community notices it, and comes after them. It isn't difficult, let some people can't even manage that.
Property owners do occasionally have their property seized by the community that it may be used more appropriately to the community's desires. It may be by eminent domain, or civil forfeiture, or adverse possession, or by regulatory taking, or some other means, but it is more likely to happen to those that do not actively demonstrate value to their community (perhaps in the form of financially supporting the local lawyering economy, if nothing else) than those who do not.
It's a fact of existence from the Pauli Exclusion Principle all the way up to "wherever you go, there you are" that everything has to have a place to be. You can't have a general welfare without a place to put it, and wherever you put it has to be adjacent to other places.
So that special obligation is basically to not abuse the inevitability of adjacency and the tyranny of distance in a way that is too obviously selfish.