I liked good ol Slashdot’s system, where if you vote, you need to select a reason with a drop down menu. This tiny bit of extra friction might be helpful to discourage quick “drive-by” downvoting and brigading. Additionally, their meta-moderation system helped to make sure people were really voting in good faith. I think they were really ahead of their time in some respects.

Also, give people a vote budget so they think about whether they really need to vote. Maybe 5 votes per day. For whatever reason I can only write 5 posts per day here, so I try to use them wisely. Same should go for votes.