Society is self-regulating, it's important to avoid herd-mentality within society, and that's why people talk about protecting free-speech, but when everyone agrees that something is not ok, e.g. sexual harrasment is not ok and shouldn't be protected by free speech, then there's really not a problem with allowing these rules to exist. A society where everything is ruled by some sacred maxims, like some sort of philosophical school, doesn't exist, life isn't that simple.