Or just make it cost karma. Anyone that down-votes too much would wind up at the threshold for down-voting and be rate limited by their ability to make up-voted comments. It would also render ineffective the strategy of having a bunch of accounts with enough karma to downvote and using those accounts all at once to make comments you don't like dead (the presence of corporate or state controlled accounts employing this strategy has been suggested multiple times).