How Not to Sort by Average Rating (2009)

Re:How Not to Sort by Average Rating (2009) by vcdimension on Thursday July 09, 2015 @08:54AM (#9857532)

For better accuracy with small samples you could use the multinomial distribution instead. The covariance matrix for the rating probabilities can be found here for example: http://www.math.wsu.edu/faculty/genz/papers/mvnsing/node8.ht... Then the variance for the expected rating can be calculated as a weighted sum of the values in the covariance matrix.

These companies really should be hiring statistics consultants instead of relying on the intuitions of their programmers.

[ Parent | on HN ]

Re:How Not to Sort by Average Rating (2009) by stdbrouw on Thursday July 09, 2015 @09:11AM (#9857612)

I'd prefer to just treat scores as continuous and correct using `t_ppf(.975, n-1)` instead of the normal approximation (1.96) but I suppose working from a multinomial distribution would give pretty similar results.

[ Parent | on HN ]

Re:How Not to Sort by Average Rating (2009) by vcdimension on Thursday July 09, 2015 @09:23AM (#9857671)

You're still relying on the central limit theorem (i.e. a reasonable amount of data) : using t instead of z just corrects for the fact that you only have sample variances instead of population variances. However, I suppose it's not unreasonable to assume that the ratings are likely to have a bell shaped distribution (which could be checked), so the normal/t approximation is probably going to be OK.

[ Parent | on HN ]

Re:How Not to Sort by Average Rating (2009) by stdbrouw on Thursday July 09, 2015 @09:33AM (#9857737)

Ah yes, true. Let's call it a bias/variance tradeoff ;-)

[ Parent | on HN ]