How Not to Sort by Average Rating (2009)

Re:How Not to Sort by Average Rating (2009) by yoz-y on Wednesday August 30, 2017 @12:56PM (#15133773)

Maybe the problem is with stars and wording. Currently most of systems are worded (e.g. amazon) in a way that 3 stars is the base and people would add stars if their expectations were exceeded and remove them if they were not met. However at all places that I have seen it is like you say 5 stars is for expectations being met and it only goes downhill from that.

I think that a wording and iconography in 4 steps could be useful. -2 = something really bad happened, -1 = below expectations, 0 = happy customer, 1 = exceeded expectations. Forcing people to write a detail on any rating other than 0 would make most ratings 0. Angry people usually like to write comments anyways.

[ Parent | on HN ]

Re:How Not to Sort by Average Rating (2009) by ghaff on Wednesday August 30, 2017 @05:44PM (#15136309)

I'm not sure an expectations-based rating system is the norm though.

To use an example I gave elsewhere. I order a cable from Amazon. It works. Therefore it met my expectations of a working cable. Yet, I think most people would interpret a 3-star rating as my being lukewarm on my purchase. I'm not. But what the heck do I expect a cable to do other than being a fair price and to work?

With respect to movies. Some movies get really built up and I go in expecting great things (e.g. Fury Road). I come out thinking they were just OK. So maybe 3 stars. But definitely not -1 or 2 stars. My personal expectations aren't necessarily a good baseline.

[ Parent | on HN ]

Re:How Not to Sort by Average Rating (2009) by yoz-y on Thursday August 31, 2017 @03:38AM (#15138766)

For a single data point they are useless. But I thing a rating in aggregate could be informative. Of course, the rating is self defeating as too many people who were positively surprised will raise the expectations.

For movies I would love to have reviews in style: Somebody writes up their expectations before going to the movie, and then rates the movie according to that. I find most "press and critics" reviews useless as if somebody is not a fan of story-less action movies, then why the hell is he reviewing them.

[ Parent | on HN ]

Re:How Not to Sort by Average Rating (2009) by enobrev on Thursday August 31, 2017 @06:31AM (#15139363)

This is how I see it as well. It would also likely get rid of comments / ratings like "I would give this 0 stars if I could". I think the numbering you mention is just as essential (especially 0).

The way I generally rate things falls in line with this idea:

5: Excellent

4: Pretty Good

3: Average; Unsurprising. Not Impressed, Not Disappointed

2: Kinda Sucks

1: Run Away

This definitely doesn't seem to be how everyone else is using the 5-star scale.

[ Parent | on HN ]