You assume one-on-one conversations. In a group setting even a strong bully can be subdued by the group. In such a group, a man isn't going to be allowed to insult a woman without other men stepping in. But you bring a good point. Should insulting, disrupting voices be tolerated in a conversation in the name of freedom of speech? I would say no, because they make further speech impossible, but I see the problem in deciding what constitutes disruption/insults.