Technically, even the notion that harm is something to be avoided is itself a moral take.

I guess a better way to phrase it is that once you start policing morality on one particular matter and create tools for that purpose, those tools will eventually be used to police morality to conform to social consensus across the board.