I think the issues you mention can be addressed. I've outlined how, below:
1: > """If you upvote a comment as interesting and the not the ones leading up to it, then you are punishing that comment because you're punishing its predecessors in the threaded chain, so it will be less-seen because you upvoted it."""
That's a good point. I'm not sure if it is an issue, however. Not upvoting a comment would have very little impact — it wouldn't weight as much as a downvote, for example. So I think the effects wouldn't be that bad. (A comments score could be: `(upvotes - downvotes) / number_of_people_who_read_but_didn't_upvote` and adding +1 to `number_of_people_who_read_but_didn't_upvote` will have only a tiny tiny effect.)
Anyway I think the issue can be addressed like so:
If there is a thread with very interesting comments somewhere in it, prioritize that thread a little bit — so it'll get a score that is a little bit better than the very first comment in it (the comment that starts the thread).
I think SlashDot does something reminiscent of this? — Sometimes, [a mediocre comment that starts a new thread] is collapsed, but [comments deeper inside the thread but with many upvotes] are shown in full.
Alternatively: Modify the algorithm: Don't punish ancestors, only siblings.
2: > """Any other assumptions of which comments are read will probably not end well. Some people may skip to comments to get someone's personal opinion, other people may only read shorter comments."""
I think it's okay that the algorithm makes mistakes sometimes. It only needs to work well on the whole, given data about many visitors. — On mobile phones, however, where only 1 comment is shown at a time, it'd be easier to know what the reader is reading.
Also keep in mind that the examples in the article were intended as examples — if there are issues, the algorithms can perhaps be modified to take them into account. For example, your example visitor that reads only short comments, would be more likely to upvote only short comments. So the algorithm could then "realize" that "oh, this visitor only reads short comments. I'll take that into account" :-)
This is also bad because, if true, it creates a strong disincentive to comment on less popular stories because, regardless of the quality of your comments, doing so would lower your average karma.
Description: http://amix.dk/blog/post/19574
Comment sorting: https://github.com/nex3/arc/blob/b78c7f/lib/news.arc#L2028-L...
It must be more than just a function of time and score, otherwise any of the new 1-point comments would have superseded mine, but they didn't.
Source: experience :)
The topic isn't so much poll position but giving every comment a chance at being noticed so that the better options rise naturally to the top and create a better conversation.
I don't think this is the "I don't care about silly internet points" discussion you think it is.
If you could collapse the top level comments, this problem would evaporate.
1 http://braythwayt.com/2012/03/29/a-womans-story.html 2 https://news.ycombinator.com/item?id=3772292
I think HN would have to increase the total number of comments per page before allowing collapsable chains.
> In the two examples above: When you upvote a comment, the computer thinks that the other comments you have read (the blue ones) but did not upvote, are not terribly interesting.
If you upvote a comment as interesting and the not the ones leading up to it, then you are punishing that comment because you're punishing its predecessors in the threaded chain, so it will be less-seen because you upvoted it.
Any other assumptions of which comments are read will probably not end well. Some people may skip to comments to get someone's personal opinion, other people may only read shorter comments.
~~~
It's worth noting that HN has an interesting algorithm that does not exactly sort by comment score. I have a very high average karma, and when I post on a topic, even if the topic already have 60 comments, my comment begins at the top and stays there even if I'm not upvoted. I suspect that the sorting algorithm is doing something like implicitly adding average karma to the karma of each post when determining order. Therefore it doesn't sort by "best", but by "well regarded".
This is good and bad. It's good because if a person who usually makes high quality comments comes along late to the game, their voice will be heard. It's bad because it creates its own rich-get-richer problem.
I think a better solution to the problem is to simply:
1) Hide upvote numbers like HN is already doing[1]
2) Dither the comments between "new" and "well regarded". First comment is the newest, second is the most well regarded, third comment is the next newest, etc. Perhaps randomize which category comes first.
~~~
[1] Of course hiding upvote numbers has its own major problem because sometimes the hard number of "I agree" upvotes is important. When you want to "Ask HN", say, what particular frameworks people can vouch for, you have no real way of evaluating the comment responses in terms of people that agree. Since they aren't even sorted from best to worst, the information you have is somewhat dim. This could be remedied by giving submitters the ability to turn on comment scores for only their thread. (Or the ability for mods to do it)