And yet, I just upvoted your comment because it seemed much easier than typing a reply to say that your comment was of interest to me and I enjoyed reading it. But now I've said that as well.
I like having the flexibility to do both, but I recognize the compelling arguments you've made about the problems with having voting available everywhere. On HN in particular, I feel the downvote capability should be further constrained than it is presently (I think it opens at 500 karma presently).
Ideally upvote would only be used for well-stated well-reasoned arguments with a plausible basis, regardless of the actual position, but I sometimes find myself upvoting a comment because I agree with it's conclusion. I think the agree-upvote is lamentable, but unavoidable in general.
Some UI changes might be able to fix the disagree-downvote (i.e. change to 'flag'), but I feel that the agree-upvote is too much a part of human nature to have an easy fix.
Voting based on insightfulness or relevance simply cannot work when only a small minority can ever be expected to stick to it. We are being conditioned on picking a side and then boosting that side no matter what.
Voting up a comment that is in disagreement with your own opinion makes it look like you are losing the debate if being upvoted is generally associated with being right. So most people won't do it.
I believe the only way to prevent this in a community is to force people to use only words to express opinions. In my view, voting has no place in debates. And yes it means that we're not going to get our debates filtered and ranked according to insightfulness. But at least we're not getting any false signals based on perverse incentives either.