I wouldn't and couldn't prevent them from talking to one of my women and convincing her that it would be a better choice to leave me and enter a relationship with them. And you could of course make the same argument for a monogamous relationship, you are still removing one potential partner for everyone else from the pool of potential partners - but you are of course not really doing this, at best temporarily. And what a single person thinks or feels never really matters anyway because a relationship is a mutual thing between at least two people.