The one thing I can't stand, and think is unfair, is when a few bad apples ruin a decentralized movement for everyone. BLM had the exact same problem, idiots delegitimizing a movement fighting a very legitimate problem, and then those scattered incidents being collected to allow organized detractors to mischaracterize them.
Now we're seeing that happen with the rightwing protests as well. Don't get me wrong, I hate nazis and also do not at all agree with Trump supporters / people that want to keep the confederate status, but they're not all nazis.
Anyway, the connections between Nazism and gamergate have been extensively detailed across the web.
But here's Stormfront endorsing the movement:
Also, not all the people protesting were chanting "Jews will not replace us". Oddly, some of the white nationalists that helped organize the protest were Jewish. And I also personally know many conservative people in favor of the statues that aren't nazis either (and at least for some, it's because they personally are Jewish, so it would make no sense).
Calling all conservatives white nationalists is the equivalent of calling all BLM protesters black nationalists, or all counter protesters antifa. It's just easy namecalling and almost always ends up just being very heavily cherrypicked images and such used to construct a narrative.
So there are limits. Even Ted Cruz will eventually stand up and say: you know what, this is too fucking far. STOP.
They key to moving the Overton window is to do it gradually. Charlottesville was a massive overreach, and it backfired.