Sure — you just deny those same rights to anyone you deem a “fascist” in a secret report. Much like say, the Stasi would allow you to speak your mind unless you were a capitalist subversive, as clearly documented in your secret trial.

Obviously we should censor fascists and subversives!