A problem with downvoting on many sites (perhaps HN to some extent) is that people seem to just use it as a generic "I don't like this" button or as part of an upvote/downvote war to make sure that their preferred comment "wins."