Have you considered the possibility that Hitler may be lying or deluded?
The Nazis had their own paramilitaries to start mob violence, and in Britain the https://en.wikipedia.org/wiki/Battle_of_Cable_Street is credited with preventing the rise of Fascism in the UK. Effectively it's where the "antifa" movement comes from.
The Nazis were continually repressed by mob violence and later by the state. Hitler specifically credits this repression as strengthening their convictions and helping them recruit.
That is, restrictions on freedom of speech led to the Nazis.