While this is more sophisticated, I doubt it's the "correct solution". The underlying assumption of independent draws from a Bernoulli distribution is vitiated, as the order in which the siblings are read usually affects their proportion of upvotes to downvotes. It still significantly penalizes late comments, because the 95% confidence interval is much wider for them (and its inferior limit consequently lower). Last but not least, it brings down comments that are at all controversial; whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval. In short, this sorting algorithm seems ideal for quickly placing on top bland comments, that nobody really disagrees with but will upvote anyway because they haven't yet seen the much more incisive discussion below.

> the 95% confidence interval is much wider for them

If you change this to an 80% confidence interval, it'd become narrower and might actually somewhat favor new comments, with only a few votes (upvotes)? So this might be configurable?

> whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval

Isn't it more likely that the 1000 upvotes and 500 downvotes is a cute kitten photo? Or something similar (a very short but strong and popular comment)? :-) And that the one that gathered 50 upvotes and no downvotes is the truly interesting read?

However! If you're able to estimate how many people actually read the comment that got 50 upvotes — then you'll know if it's truly interesting, but too long to read — only 50 people have read it. Or if it's boring — 1500 people have read it.

Perhaps a good sorting algorithm would be: Lower bound of confidence interval for:

(upvotes - downvotes) / num-people-who-read-the-comment

The problem is that kittens are quite popular. They are more popular than a well formed discussion. What algorithm separates kitten posts from quality content? How could you tell the difference between a great photo and a lolcat? I think the only way to do this is to empower moderators. Any other scheme is just a race to the middle for bland, unoffensive and easily digestible content to maximize points.

Letting different comments reach the top wouldn't be such a bad thing most of the time, as long as there is more than 1 solid comment.

Only time this would be bad is if there is something important, such as a correction or a key reply by the author or the person they were calling out etc where it makes sense that everyone should read first.

Knowing how many read a comment would help with comment assessment but how do you get that information. I suppose we could track scroll positions, I'm not sure how reliable it would be.

It'd be cool if websites could use eye tracking, then we could easily tell what got read. Maybe in the future.

Wilson score confidence interval is easy to implement and already proven to work greatly on Reddit, where it is the default sorting algorithm. Do note that Reddit also has down votes and Reddit also has threaded comments. While I agree, it's not the "perfect" solution, but for a perfect solution HN admins would need to invest a lot of time (and it would probably be a lot more buggy and in the end they probably won't come up with something that's better). Going with Wilson they can do a quick fix that would improve the sorting a lot.

I can't speak for anyone else, but I've been consistently more happy with the comment rankings on HN as opposed to Reddit. This could be something to do with voting behavior, but I think it's at least partly thanks to HN's ranking algorithm as opposed to Reddit's.

I'm actually using the algorithm that Evan Miller writes about (i.e. the lower bound of a binomial proportion confidence interval), in the discussion system that powers the demo in the linked article. I'm using the Agresti-Coull interval, http://en.wikipedia.org/wiki/Binomial_proportion_confidence_..., however, because it seemed easier to implement. (I didn't know about Evan Miller's article (or any other).)

For instance?

So, there are some that require offline calculations (I can't really get into more than this for various reasons).

Perhaps I'm a bit dense today, but why are you unable to link to descriptions of one or more of these better approaches? Many people would find the information useful and interesting.

Not even a Wikipedia link? This was so anti-climactic.

Wikipedia does not always have the answers :)

Sorry, I fully intended to publish a description, but it turns out at least two of them are the subject of pending patent apps that haven't been published yet, so I can't.

In any case, do note there are a wide variety of wilson variants or alternatives that may be applicable to various situations (agresti-coull, etc), as well as jeffrey's prior bayesian intervals.

http://stat.wharton.upenn.edu/~tcai/paper/Binomial-Annals.pd... is a reasonable comparison of the current common methods.

Anyway, I deleted the original comment since it doesn't add anything now.