If implies who, as it gives negotiating leverage. But that's kind of beside the point.

Edit: I just realized I probably misunderstood you. The point about neighbors is meant to be relevant to the notion of the government having a degree of control over your property in general, not the specific question of who's allowed to stay there. Most of my response is no longer relevant with that clarification, so I've removed it.