I tend to upvote anything that seems coherent and interesting, not just the stuff I agree with. If I ever get to the point where I'm allowed to downvote, I'll exercise that privilege only to mod down trolls, spammers, or very bad style. Everything else can much better be handled by replying to an entry and having an actual discussion.
So am I mistaken? What is the mod system for, officially? Quality control or consensus building?
This leads to cases where coherent and interesting comments are sometimes -4, which is exactly the rating that should be reserved for porn, spam and really gross trolling. At the same time, some users achieve consistently high ratings for comments that seem to be lacking in content, leading me as a new user to suspect that the system is either rigged in some way or at least very susceptible to ridiculous pile-on effects.
The good news is though that this doesn't happen often enough to be a serious problem (yet?).
I preferred to absent myself from here for a while after it got real bad (to the point of the first article commonly being a 2-3 year old essay that had bubbled to the top AGAIN), simply because I felt fundamentally outside of the norm here - I rarely agree with anything PG says in its entirety. Paul is a thoughtful enough person that I've never felt that it would upset him if I disagreed here, but I would imagine there would be a lot of people keen to argue with me until I was blue in the face about it.
An opinion is an opinion however, and we're all entitled to our own - I take downvotes not as a personal attack, but simply an expression of disagreement, be it with your opinion OR the way you express it.