If we're going to tolerate Nazi and Nazi-like speech, don't we also have to tolerate those who would should them down?
Until we start rounding up Nazis, and Nazi sympathisers, and politely asking them to reside in this here concentration camp and eat this rotten-fish-head-with-cabbage soup, shouting them down seems quite reasonable.
You're thinking about it a little backwards - here's a fascinating example of the paradox of tolerance:
Proposition 8, in California (2008)[1] was a ballet initiative that proposed that "only marriage between a man and a woman is valid or recognized in California".
Proposition 8 passed and the post mortem indicated that black and latino/hispanic voters (who are not politically progressive in the manner of coastal californians) voted (overwhelmingly, in the case of blacks) in favor of the proposition - so much so as to be a deciding factor in its passage.[2][3][4]
[1] https://en.wikipedia.org/wiki/California_Proposition_8_(2008...
[2] http://latimesblogs.latimes.com/lanow/2008/11/70-of-african-...
[3] http://www.washingtontimes.com/news/2008/nov/8/blacks-hispan...
[4] https://www.washingtonpost.com/news/the-fix/wp/2013/03/26/ho...
The minorities in question were welcomed into society and empowered with votes then promptly voted to oppress another minority. There are clearly problems with this.
What are we to do? One way of dealing with the problem would be to ask the question at census time: Do you believe in Nazi or Nazi like ideologies? and then round those people up and put them to death. This has it's own problems.
Doing things like clamping down speech that incites violence or otherwise directly harms people is working. More options are open for almost all members of minorities than ever before (in terms of jobs, opportunities, legal recourse, etc... ). The nazis are just being loud about it now because they think trump will somehow protect them.
Convict and arrest people advocating for the death's of others, financially fine those empowering that one specific kind of speech and the people will continue to get smaller. Without the violence hate speech cannot grow.
There are other options, reasonable limits, as you've pointed out.
We can go with the data.
To me it seems like the opposite is happening. Nazi ideologies are marginalized and meaningless, yet propped up like some huge threat to us... like they might take over. And this is being used as justification for limiting free speech, in a hyper-politically correct society where even micro-aggressions could be categorized as "hate speech."
Perhaps we could just recognize and acknowledge that, while reprehensible, it's not like Nazi believers are anything more than a rounding error in normal society. And then move on with our days.