I usually downvote comments that are derailing the thread or are so lazy as to feel like a waste of my time to have read.

I'll often upvote comments with opinions that are counter to my own, especially if they are presented in a clear, coherent, or novel way that causes me to think a bit.