Exactly. I get that that there are plenty of comments or threads that are so "wrong" that you don't want to dignify them with a response, but usually my reaction to those is to press the [-] and hide it. For the other cases, I check if there is any response already that addresses the problematic points and try to not just pile on the downvotes.

What I don't get is when I see a totally grayed out comment with a valid argument but no response whatsoever. These seem like they were just the victim of a mob attack that wanted to shut the conversation out.