Well, I would argue that a good balance is to hear people out regardless of whether they are "literal Nazis", but reject "Nazi speech" (such as advocating for racial violence) regardless of who is spouting it.