It always comes down to censoring those you disagree with. Everyone loves free speech when it expresses views they agree with. If it's about silly edge cases, no-one really cares - the only reason to care is if it has political implications, regardless of if you are for or against suppression.
I don’t think I even care, but please be honest about it and tell people why and what you would ban.
I am sure that, like every one using the arguments you are using, you would suppress free speech much more than that!
Be open and direct, we still have free speech my friend!
No need to hide between rhetorical devices.