Don't know how much that applies here, but there is no inherent moral or egalitarian negative to a "harem" (or "unicorn" arrangements if I remember my polyamory lingo correctly) if everyone involved is actively and consciously consenting. But Gandhi is probably not in that camp