I agree, downvotes are dangerous. I know Reddit had (has? I don't know I left when I found HN) a problem where all new articles were getting downmodded by people who wanted their own links on the front page.

I repeatedly think that I would like downvotes, but I quite quickly remember that I absolutely hate downvotes. I don't completely agree with being able to downmod comments, however I think the side effects are dampened that you have to have good karma before you can downvote, which prevents trolls from abusing the system.