This really does bug me. The whole "if-you-downvote-me-you're-a-jerk" sorta makes me want to, even if the comment itself has some kind of decent reasoning. With that said, i can also be frustrating when people downvote to invisibility things that are not wrong but with which they disagree.