News for hackers. Stuff that matters.
Re:Why the 5 Star Rating System Isn't Suited to Local Reviews by apinstein on Sunday May 03, 2009 @08:58PM (#591671)
This article seems kinda troll-y to me.

Why? Because even Yelp understands the "The man who drowned in the river whose average depth was 6 inches" problem, which is exactly why they offer both a histogram view of ratings, and an average-over-time graph. I frequently use both of those tools.

Now don't get me wrong there are all kinds of problems with 1-5 rating schemes, and there are probably better schemes out there. But what Yelp does is as good as any I've seen towards combatting the problem, which is why I enjoy using it.

As someone that took 4 semesters of business statistics in college, my personal favorite alternative scheme is paired-comparison, or even better yet, the related (and newer) MaxDiff algorithm [http://en.wikipedia.org/wiki/MaxDiff]. The only problem with these schemes are that they require much more user input which frankly is a lot to ask for on a site like Yelp.

Re:Why the 5 Star Rating System Isn't Suited to Local Reviews by pmjoyce on Monday May 04, 2009 @05:28AM (#592153)
I agree advanced statistical analysis is not suited to these types of sites as the requirement on the user is such that the response rate would be virtually nil.

But the point is that while the 5-star system initially looks and feels like it's offering some real data with which to evaluate a bar/restaurant, the reality is that the data need advanced interpretation to come even close to being meaningful not to mind useful.

There's probably not a perfect system out there but I'm pretty sure that other models would be more "honest" at least.

Re:Why the 5 Star Rating System Isn't Suited to Local Reviews by thwarted on Monday May 04, 2009 @02:18PM (#592784)
The 5-star rating system is exactly the same as asking someone "would you use/watch/go/buy-from this product/place/movie/business again?". You ask a few people and you consolidate their answers into a aggregate perception and then make your own decision based on that. Then you go there (or not) and make your own judgement and add your input to the pool. This is exactly useful for the intended purpose and audience.
Re:Why the 5 Star Rating System Isn't Suited to Local Reviews by pmjoyce on Monday May 04, 2009 @04:17PM (#592985)
I disagree strongly with your statement that the 5 star system is "exactly the same as asking someone "would you use/watch/go/buy-from this product/place/movie/business again?". "

In fact, I think this is exactly where the complication arises from. I agree that that is probably a close approximation of the question one would like to have answered, but asking people to give a rating out of 5 stars certainly doesn't ask it. Not only that but it actually muddies the water somewhat when aggregate data is spat out as an answer.

Re:Why the 5 Star Rating System Isn't Suited to Local Reviews by thwarted on Monday May 04, 2009 @06:10PM (#593185)
Agreed, that is a question that needs a boolean answer, and a rating scale is a range of answers. But if you were to ask ten people that question and their average answers came back at above the half-way point, would you think it was a good place or not? If nine of your friends said they had a good experience and one said they didn't, how is it not that this place is considered to be a good? That's all these rating system strive to do.

I also don't see how any of this leads to less or more "honesty" in the ratings.