Goodfilms (goodfil.ms), rest in peace, had a great two-rating system with Quality and Rewatchability, because the latter turned out to be a really useful metric.

https://news.ycombinator.com/item?id=3082241

One of the main frustrations I have with Goodreads is how limited the API is nowadays, and how there appear to be no measures against brigading and other campaigns. One of the core issues with ratings services.

Personally I'm hoping Open Library by the Internet Archive grows more in popularity, given how most websites come and go:

https://openlibrary.org

I'm curious—what would you like to do with the API?

Ironically, mainly to check for irregular voting patterns like brigading.

Not dissimilar to what Steam implemented, which is basically Bollinger bands for ratings.

How does one use an average Rewatchability score to determine whether one should watch a movie?

If I'm trying to pick a movie, I don't care what its score for rewatching is, I care about what its score is for watching it the first time.

And once I've watched a movie, I don't care about whether other people say I should watch it again, I care about whether I want to watch it again.

A movie is different from buying a board game. If I'm shelling out $50 for a game, I'll want to know if it's still fun the twentieth time I play. But that isn't a consideration when picking a new movie to watch, the experience may be worth it even if I never watch it again. And ditto with books. I'm probably not going to read that 800-page book again, but that shouldn't stop me reading it once.

To present an opposite viewpoint, I try to only engage with media I suspect future me will be able to revisit and pull enjoyment out of at least once. I rarely actually do so, but I've found it to be a remarkably effective quality filter.

It's also a genre independent quality metric. That's not to be underplayed. Some examples of films that successfully passed it for me: Casablanca; Portrait of a Lady on Fire; Hereditary; Under the Skin; My Neighbor Totoro; The Fifth Element. I'm pretty sure most people would agree at least half of these movies are good.

It turns out most of the things I consider worth revisiting at least once are also things other people would consider that way. So for me a Rewatchability rating is a positive signal.

Have you ever watched a film like Come and See, Schindlers list, Grave of the fireflies, etc…?

These movies are some of the most impactful that I have ever seen, but by no means would I rate them highly rewatchable. They are gut wrenching, and some people can only stand to watch them once, few want to rewatch them, but they are also incredible.

I also gain a lot from a rewatchable piece of content, but you might be shorting yourself by always watching things designed to tickle the dopamine receptors.

I've seen the latter two, but they are most certainly very rewatchable. I saw Schindler's List again with my wife about a year ago and enjoyed it just as thoroughly as the first time.

>I also gain a lot from a rewatchable piece of content, but you might be shorting yourself by always watching things designed to tickle the dopamine receptors.

Now this I just do not understand. Things designed to be good on primarily the first watch, and allowed to degrade on future experiences, seem much worse for this.

Ahh.

I think we are coming from different feelings about rewatchability.

If you asked me to rate movies as to their artistic merit, their excellence as films, I would say that those all fall into “instant classic” territory. However, I would not want to rewatch them in the same way that I might want to rewatch a Coen brothers film, for example.

I agree! I think this is one reason why Rewatchability is an imperfect, though positive, signal - many people mean it in the sense you mean it, and would therefore mark them as not very rewatchable.

That's fine by me, of course. The more signals I have, the better my decision can be made on what to watch next on average.

I feel like a movie can be a good time, but I wouldn't rewatch it.

For example, I just watched the Gorge. I enjoyed it, but I wouldn't rewatch it. I don't think it necessarily deserves a bad grade though.

Now, some great movies I wouldn't rewatch. La vita e bella and grave of the firefly are beautiful, I just cannot rewatch them.

It's definitely a limited metric tho.

Agreed in general that 10 and even 5 is too much, and that 4 is a good compromise. Though personally I prefer thumbs up and thumbs down, plus a separate starring option. The first two signify “would I recommend this to anyone else” while the latter means “this has something interesting I’d like to revisit at a later date”. Something losing its star rating is par for the course, but the recommendation status is less likely to change (though it can happen). And yes, it is possible to give something a thumbs down and favourite it, e.g. when you don’t think something is particularly good or competent but it still had something which you recognise as meaningful to yourself specifically.

I don’t think this system is right for everyone, but I like it. Depending on the platform I may even use a rating system of 1, which represents the starring and everything else is just read/watched.

I informally often use a -1 to 2 scale. Bad, fine, good, great.

The difference between 1 and 2 on a 5 point scale is not useful.

This is a great scale to use when you’ve got a group of people because it’s easy to teach without needing to do calibration: -1 is bad, not meeting expectations, 0 is “not good, but meets expectations”, 1 is “good, better than expectations” and 2 is outstanding.

It’s hard to come up with a nice visual for it though, you just have to use the numbers themselves (or rather ugly emojis)

> It’s hard to come up with a nice visual for it though

Suggestion off the top of my head: down arrow / thumb down; circle / horizontal dash; up arrow / thumb up; star / heart.

The circle / horizontal dash could be ambiguous in isolation but should be clear in context.

No, and that's why Netflix switched to thumbs up/down instead of 5 stars.

And they mostly ignore that. They have access to what you actually watch, rather than what you say you like. They know what you start, what you finish, and how quickly you jump on a new one.

It doesn't matter how many times you down vote Mexican soap operas or singing talent shows. If you keep watching they're gonna keep suggesting them.

And result is stupid. It is pretty much impossible to discover something new on Netflix. It puts you into a quick box and no matter how much I try to find a comedy, I can't because it decided the same crime shows are my thing.

Netflix recommendation system just does not work. It does not allow me to find movies I can like, it allows me to see the same thing I seen once before.

I am not in the mood for serious documentary evey day of course, it takes more concentration. But when I am in mood for one, I should be able to find it.

Yeah and I hate this scheme. Facebook keeps showing me clips that I do not want to watch and yet fined myself drawn to watching like some compulsive behaviour. Urgh.

It's debatable this was the motivation for Netflix. More stars ==> more nuanced ==> more qualitative reviews ==> much more effort and time to decide if you should consume. Netflix is long(er) form TicTok and wants to optimize for continued consumption without friction. I wouldn't be surprised if they drop ratings all together and only offer a personalized AI curator stream. They could do this just based on viewing time and engagement and avoid even the minor disruption of "up/down". Don't make the sheep think.

They switched because it took more effort from users to rate on a 5 vs just saying good or bad. Because Netflix is a streaming service, casual users don't want to put in that much effort when they're chilling. Tracking is fundamentally different, where you go in with the expectation of organizing your library.

Last time I saw a movie there there was "super thumbs up" in addition to normal one.

For clarity I'd replace rating systems with "was it a good spend of my time?" yes/no question. Then just show percentages. Could not be clearer.

Having worked on a rating system, I think 10 levels are useful (or 5 levels with half-stars). You can create better averages/recs using 10 levels, since ratings of 0 or 10, which are often spammy, can be down-weighed.

I can absolutely understand that it makes sense to have a much more fine-grained average, but personally I struggle to give meaningful ratings beyond the four I mentioned.

That said perhaps multiple binary dimensions would be better. Good story yes/no, interesting/unique premise yes/no, overall good acting yes/no, good cinematography yes/no etc etc.

I was interrupted, so when I picked up my brain continued with movie recommendation mode, I watch more movies than read books, but the binary multi-dimensional rating could of course be applied to books as well.

I once had an interesting conversation with my mother in law who was telling us about how much thought and effort she puts into scoring shows on IMDB. She was shocked to learn that I want my opinion to have more impact, so If I like a show I give 10 and if I didn't it gets 1.

Yes, but sadly a 10 is often used by fans of a lead actor to pump the ratings, and a 0/1 is used by the moral police as a protest vote.

It just feels satisfying to me—organizing my library neatly by rating, based on how much I like each book. A four-level system just doesn’t have the same impact. Personal preference, I guess.

Just ranking everything in order is the ideal rating system. And it's easy to convert to whatever poorly-chosen system someone else wants you to use. It does mean the scores for your first few ratings will fluctuate quite a bit, though!

If you added half stars and stuck to a 5-star system, there's less user surprise for functionally the same thing.

I did go back and forth on this. I just like 10 stars better. IMDB, MAL, Mangadex all have 10 stars. So it's not really uncommon. But yeah, if a lot of people feel 5 is enough, we'll just return it back to the 5 star system.

Anilist let's you switch your rating system. You can do 5 stars, 10 points, 100 points or a simple 3 points reprrsented with emojis. I quite like the idea, although I think it makes it a little more complicated to calculate ratings for all people.

You could do a simple weighting, probably adjusting for sample distributions.

I would guess that with experience and age preferences change. What's the point of a 10 star rating system if you have a 10 star rated book that you wouldn't rate the same after being ten years older?

Rating is a mess. Still I think people are used to express their opinions on a 0-10 scale, and they will appreciate if they can just keep doing that.

0 to 10 is a mess as well because some people give 10s very often and others only give a 10 for the very best things. And of course the bots all rate everything 1 or 10.

I was once playing around with a 3-point ranking system. Think thumbs-down, thumbs-up, and double-thumbs-up. The thumbs up and down would basically function as expected, while the latter would be weighted heavier for the recommendation algorithm. Basically a `recommend me more of this, this is high quality content` action.

There is a general problem with a 5 or 10 star voting system, consider a [malicious] user who only gives a 1 or 10 star vote, thus ending up with more voice than one that votes in the range of 4-6 which would be what the majority of the content deserve. Therein lies another problem too, while the scale would imply 5.5 to be average [out of 1-10 with no 0 option], most humans tend to consider 7-7.5 to be average instead, there's a very natural bias on the scale.

This idea isn't actually uncommon however, as platforms tend to work with a thumbs-up, thumbs-down, and a `favorite` action of sorts. Some platforms tend to respect favorites in recommendations and some don't. I have found that YouTube doesn't care all that much about my... let alone favorites, it doesn't even care about my votes. TikTok however did this well, I had downloaded it one day and at the end of the day my feed consisted of neat programming tricks and lessons on color theory. Which kind of revealed something my own prejudice too, as I had expected TikTok to show me the worst content and it was the platform that respected my choice the most. That said these things change a lot so it wouldn't surprise me if the same test shows the opposite results a year from now on.

What you're describing is fully observed in education. The scale of your evaluation has to have an odd total of numbers and a limited number of choices. 1-2-3, for example. That means that each digit sends a strong signal. 1-2-3-4 means the 3 is the non-controversial choice. Average, so to speak, and you don't want average in your evaluation. You need to adequately grade stuff here, and giving most things an average grade is a weak signal that prevents you from differentiating.

When you grade by competency (not by knowledge), you also assign a written description for each grade. That helps a lot. I think those platforms are keenly aware of those facts I just described, and are trying to boil them down to simple actions for users, that impart large signal, and that respect the cultural norms of evaluation. That's why Letterboxd has a 5-star with half-stars rating system, but also has a like button.

for me there are 3 levels:

- books i wish i hadn't lost time with

- books i've read and were probably ok

- books i would give/recommend

I could see "proportion of 10s" vs "proportion of 5s" (the max score in either case) being meaningful.

Should be Positive / Neutral / Negative and anything beyond that described in text.

Reasoning: number ratings are subjective. My 4/5 is not the same as yours... or even the same as mine 2 years ago.

Sure, but can we really deny that the 5 star rating is pretty decent and simple, having an exact middle point and then two levels for each side.

Well yeah, but I feel the neutral midpoint is problematic because I feel it's too easy to pick it. Removing the midpoint forces one to decide which side it leans towards.

If one really feels the need for the "meh" category, I'd say go for a 3-level system: bad, meh, great.

Black Panther has a rating of 96% at Rotten Tomatoes. So whatever the scale is, nothing is really meaningful. I've been in the cinema with about 60 other people and a friend. statistically one of us should have enjoyed it but we both found the movie shit. So no, 10 stars is not more meaningful as 5 stars. 2-4 should be enough to still be wrong about taste once in a while.

96% means that 96% of critics liked the movie enough to recommend it.

It also means that 4% of critics did not recommend the movie. In a theater of 60 people, you and your friend would fit into that 4%. So there's nothing wrong with RottenTomatoes.

> Does one really get anything meaningful out of saying this was a 6-star book vs a 7-star book?

I rate for myself, and not others. And for over 20 years I've used a 10 point system.

10 = Easily amongst my favorite

9 = Awesome, but not in all time favorites

8 = Really liked it, and would recommend

7 = Liked it, was worth my time, but not so much that I would happily recommend to others

6 = Liked it, but wasn't worth my time

5 = Neutral

And below 5 I don't distinguish. I randomly pick to indicate I didn't like it.

It kind of seems like you use a five-point system, then?

The difference being that I don't want someone to think that because I rated something 2 out of 5 that I didn't like it because it's less than 2.5. And for most people, a 3/5 doesn't mean "yes, I heartily recommend". But they can believe it for 8/10

(Actually, 7 points as someone else pointed out - by my point stands even with a 7 point system).

you use a 7 point system

I agree with a lot of your top 5 so I’ll try to continue

5: I enjoyed sections of this book but as a whole I didn’t like it

4: had some cool ideas and there were moments when I got excited but the execution wasn’t there. Basically an amateur with a good idea

3: readable but unsatisfying. I finished it but was roasting it in my head the whole time

2: garbage. Bad story idea and bad writing. Nothing good to say except that it seemed like the author was trying

1: offensive. Celebrity cash grabs, polemics, etc. no artistic value whatsoever, author was not trying to write a good book. “Book” is just a format here

The thing is, 10 means "I'm a fan" It doesn't tell anything about if I've liked it or even if I've watched it

The 4-star system is indeed wonderful. A rating that is truly "neutral" doesn't exist in a world of human subjectivity

I completely agree. I've never read a 1-star (to me) book because that implies it's unreadable, and anything good enough to keep my attention room is generally 4-stars and rarely 5-stars. I bet if I look at my Goodreads it's 60% 4s, 30% 3s, and 10% 5s

I don't rate books on Goodreads, but I do look at the average rating when deciding to read a book. I won't start anything less than about 3.7 or 3.8.

Rating books on an absolute scale makes little sense to me.

The actual questions is: Whom can you recommend this book? Even mediocre books can be very useful for the right people.

If there's no voting data, it doesn't matter what scale is used.

I typically like 7 point scales (like this[1] post from jgc outlines), but your classification also seems pretty good.

[1]: https://blog.jgc.org/2007/12/seven-point-scale.html

My stars are as follows:

5 - This book was so good that it’s life-changing

4 - This is a really good book

3 - I enjoyed this book, it was good.

2 - It’s alright.

1 - I hated this book with every fiber of my being, because it somehow tricked me into finishing it despite my hatred of it.

YouTube changed to thumbs up/down from 5-star rating system many years ago like Netflix. They learnt most people either give it 1 star or 5 stars. It’s also hard to understand the meaning of 2,3, and 4 stars in the context of a video.

No, that's meaningless precision. meaningful precision would be stars by category, not a generic overall assessment

I don't feel it's entirely meaningless, but I do agree multidimensional rating would be more informative and better to guide others.

I think one of the most robust rating systems I've worked with is based on comparisons, i.e. "did you like this more or less than X". This is more computationally intensive, but it can be made to work even with intransitive and unreliable judgments.

To me, even 5-star system is broken by design, with a pressure to rate everything 4 and 5, especially if these are known to be classic books.

I rarely give 1 or 2 - in vast majority of cases it means I stop reading them, out of respect for my time.

What is nice, but underused (since most platforms want us to be excited, because of sales and adverts) is some kind of slider with mean at 0, for expected quality.

Even better, tags to choose from "awesome", "insightful", "well-researched", "funny", "cringe", "inaccurate" etc. I mean, there are tags, but I mean ones explicitly displayed next to rating.

I think for a personal level, everything with more than 3 rating-states is overrated. Either you like it, dislike it, or don't care. Nothing more necessary. But on a social site, more nuance can be valuable to get a more fine-tuned result without polishing algorithms.

I can definitely say there is a difference between 7-star and 6-star movies on IMDb. A 7-star movie on IMDb is worth watching most of the time, while a 6-star movie is a 50/50 chance. A 5-star and less movie is a hard pass.

Ya i do love, like, meh, dislike :). I want personal ratings not a weird public scale that everyone uses differently.