That said, upvotes per page-load-including-the-comment is probably still better than the raw number of upvotes.
The number of votes an article has and the number of comments are both going to be strongly correlated with the number of views per comment. So would the mean and range of the comment scores. You could also take the time since the comment was posted into account.
You could have the visible "raw score" for each comment and an invisible "quality adjusted score" that is used to give each user a quality rating.
I think this is a feature, generally, but I get what you're saying. With any criteria you present for deciding what is worth posting you run the risk of it being the only criteria. Ideally, we'd all only care about our karma insofar as it helps us avoid making lousy posts, and ignore it entirely otherwise, but idealism doesn't scale.
"How will this influence my average?" is definitely an oversimplified heuristic for deciding what is worth contributing, but it's reasonably accurate and I think it's the best one we've got. (Better than total karma at the very least.)
As I said in the above comment, the voting on comments is the closest thing we have to an indication of their quality. It obviously doesn't always reflect reality, but what that points to is that some comments--the ones that we want around here but that would bring your average down--are undervalued. Those comments should result in positive feedback (via average karma or otherwise), but they don't.
That is to say: The site has a reward mechanism (karma). There is desirable behavior that is not being rewarded or is under-rewarded (e.g. replying to less popular Ask HN posts). Is there some way to reward that desirable behavior?
Comments which few people will notice are also discouraged. Most of my 1-point comments are the only comment on their respective articles, and many of those are 'Ask HN' posts. I don't want to find myself thinking "I could contribute here, but doing so will drag my average down".