That's probably the biggest disadvantage of the simple upvote/downvote moderation system. This is one area where I miss Slashdot's moderation and meta-moderatiom system (though I don't miss the continually degrading quality of the discussions).

It's a bit like the distinction between true democracy and a democratic republic. True democracy is mob rule. Democratic republics are slightly better at maintaining some sanity and objectivity.