Re:How Not to Sort by Average Rating (2009)
by bllguo on Wednesday August 30, 2017 @05:41PM
(#15136287)
unfortunately sample sizes will probably decrease drastically when you're looking at the subset of people who rated 4 specific games, or other such cases
Re:How Not to Sort by Average Rating (2009)
by fnordian_slip on Thursday August 31, 2017 @06:36AM
(#15139385)
Yes, that alone is probably enough to make any system like that come with a huge false-positive rate.
I myself only rarely post a review, and I could easily see how - in a vacuum - my star ratings might look like those of a bot.
Say there are 4 games. Most users who rate all 4 rate them similar, except for the 4th game that always gets really low. So 5,5,5,1 is a normal expected rating, but 1,1,1,5 isn't. So 5,5,4,4 from a high rater or 2,2,2,2 from a low rater would be given more weight than a 1,1,1,5. Other things can be added such as weighting a user's ratings a low impact if they have too few scores to determine ratings from.
This reminds me of the problem of determining the answer key to a multiple choice test given only the answers of the test takers.