One time in the streets of New York there was some racist shouting things at people of a certain ethnicity, picking them out from the crowd and calling them 'baby rapers'. Several people of different races shouted at him to "shut the fuck up". Those people obviously weren't swayed into thinking that all people of that specific race were indeed 'baby rapers'.
I would rather live in a society where people are allowed to say that the holocaust didn't happen and have them be judged as the idiots they are, than a society where saying things that are deemed as 'extreme' by the government is illegal.
I'm not saying I like it, but just saying "well terrorism isn't a threat" makes as much sense as saying "no one's every robbed our bank, so we can stop locking the door and having a guard posted outside" without giving further context.
Let's face it: Terrorism is really a minor threat to almost all people's safety, yet it's constantly evoked to further restrict people's liberties.