Perhaps the downvote button should be changed to a flag (essentially how I use it) which might discourage using it for disagreement. I only downvote a comment when it is abusive, belligerent, blatantly condescending, or clearly nonsensical and unwilling to actually converse.
Ideally upvote would only be used for well-stated well-reasoned arguments with a plausible basis, regardless of the actual position, but I sometimes find myself upvoting a comment because I agree with it's conclusion. I think the agree-upvote is lamentable, but unavoidable in general.
Some UI changes might be able to fix the disagree-downvote (i.e. change to 'flag'), but I feel that the agree-upvote is too much a part of human nature to have an easy fix.
And yet, I just upvoted your comment because it seemed much easier than typing a reply to say that your comment was of interest to me and I enjoyed reading it. But now I've said that as well.
I like having the flexibility to do both, but I recognize the compelling arguments you've made about the problems with having voting available everywhere. On HN in particular, I feel the downvote capability should be further constrained than it is presently (I think it opens at 500 karma presently).