News for hackers. Stuff that matters.
Re:How Not To Sort By Average Rating by metaxyy on Tuesday April 03, 2012 @11:01AM (#3793182)
Very interesting. So would you say developers should probably use the incomplete beta function, rather than Ev's method? Or is it too computationally expensive?
Re:How Not To Sort By Average Rating by a1k0n on Tuesday April 03, 2012 @11:13AM (#3793235)
I haven't investigated it in depth -- quadrature over a single variable as this is can be pretty quick to compute. Not sure how scipy does it.

Anyway, I personally think 95% confidence intervals are a crutch. The correct Bayesian approach is to consider two items, each with their own up and down votes, and integrate over all possible values for p1 and p2 (being the underlying probabilities of upvotes for item 1 and 2, respectively) over the observed data, and compute the likelihood of superiority of p1 over p2.

How to turn that into an actual ranking function? No idea. I doubt it would work, but you could compute against a benchmark distribution (i.e. the uniform 0-1 distribution).

If you do that, it probably turns out that your ranking function is the mean of the Beta distribution, which is simple: (U+1)/(U+D+2) where U and D are the upvote/downvote counts [note: we started with the prior assumption that p could be anywhere between 0 and 1, uniformly]. Basically, the counts shrink towards 1/2 by 1. This is a hell of a lot less complicated, and it achieves the goal of ranking different items by votes pretty well with more votes being better.