One can criticize both groups and still not be neutral. As I have stated, I do not like the neo nazis (not sure if all the original protesters are that or if only a few of them are, either way, they all seem quite unlikable).
What I have really been against is the "just arrest people for being seen in a group" which is what my comment was really replying to. Asking for discriminatory laws like this is backwards, harking back to when there were discriminatory laws against people with black skin. Although I know this is not what you were replying to, this is just to show why I made the comment in the first place. Also I seem to be being called a libertarian just for disagreeing with what I replied to, I don't think I have ever actually agreed with anything that someone calling themselves a libertarian has said, that I remember anyway.
Given such a protest–and even if you disagree with the above, please entertain this as a hypothetical–what would be the makeup of the group of people opposing such a protest? It seems to me that, in principle, everybody who disagrees with the far-right ideology of these protesters could, or even should, be among the counter-protesters. You can be a Nazi, or you can be against Nazis. But I'm having a hard time imagining someone being neutral: "I think the idea of sending the jews to the gas chamber has potential, but I will reserve judgement until I have studied it in more detail" just doesn't seem like a common opinion.
And that's why people are so outraged with the President's "there are always two sides" equivocation: one side wants genocide, the other wants "no genocide". Even if both sides had been similarly violent (which they were not: only one committed a terror attack killing someone), they aren't comparable. Because for these Nazis, the opposition is in the way of their fantasy of a whites-only country, whereas for these opponents, the step after keeping the upper hand against the Nazis is "going home".