Perhaps if the root issues were given some easy way for one to satisfactorily resolve when they arise then their would be far less support for outright bans.
The problem is that any law or rule is only as good as its enforcement. People forget this too often when writing regulations, ending up with rules that are too costly, or involve too much invasion of privacy to get right.
I think the lesson is that you should always consider enforcement when writing policy. It needs to be front-and-center, not a secondary concern. A worse policy may end up being better if it's more easily enforceable.
On the other hand, if they behave themselves just as well as long-term owners, then no, you really shouldn't have any say whatsoever about who lives in your neighbor's property.
I don't object to you running a hotel next door, but I expect your guests to behave themselves well..