Saying "Gays should be treated the same as straights by the law" isn't necessarily violent.

Saying "You can't refuse to sell a cake to a gay person" is violent, because it implies forcing a person to sell something.

What I'm criticizing is banning speech because it's "violent" while at the same time allowing other forms of violent speech. You're not banning it because it's violent, you're banning it because you don't agree with it.

Am I pushing this violence thing to the limits? Kind of, I can see a counter-argument saying that I'm being overly literal, etc. but hear this: Many of these moral inquisitors that want to ban violent speech have ideas that are actually violent, not only in the "philosophical"/overly literal sense that I described, but ideas that actually would affect good people negatively, and many times they don't only want to ban nazis and that sort of stuff, sometimes they want to censor liberal (in the British sense of the word) and conservative ideas that are held by a non-negligible amount of "regular" people.

So it's important to spot these lies when they start, even if they seem like edge cases and overly literal interpretations, I think it's good to deconstruct these ideas to their fundamental concepts and analyze them before saying "Yeah, they're Nazis anyway, ban them".