Lots of extremists, especially in the authoritarian[0] or right-wing edges of the political compass, practice various forms of censorship. This isn't merely "let's pass a law to make it illegal to say a thing", but would also include things like harassing other people in public forae with sock-puppets, ballot-stuffing online polls to make their side look more publicly favored, flooding websites with expensive HTTP requests (DDoSing), or publishing personal or hidden information in an attempt to scare someone into not speaking (doxxing). All of the above behaviors should be considered just as censorious as vanilla-flavor state-actors censorship.

Furthermore, because these behaviors nominally involve something that resembles an act of speech, people occasionally try to defend said acts on "free speech" grounds and call the curtailment of censorship acts itself a form of censorship. This is a mistake. Fringe extremists are not merely "spreading their ideas", they are chilling other people's speech. This is just as much of a danger to society as the banning of other people's views that you mentioned.

[0] auth-left inclusive, fuck tankies