The problem here is feedback. The higher rated ones get higher rated, so more people see it so it gets higher rated. That opens up a whole extra can of worms you don't want to deal with.

This is exactly true. You gotta balance freshness, quality, and uncertainty.

I agree. It's a good solution for all cases where the intent is to have a negative vote exactly cancel out a positive.

The method in the article combines a quality rating with a quantity rating, but it's a bit unwieldy and difficult to tune intuitively. It seems to me for a lot of purposes you might get a sufficiently similar effect by using method #1, and then multiplying the result with the sigmoid function applied to the ratio. The advantage of this would be you that the only magical numbers in the formula would be tuning factors you put in yourself.

This seems more appealing to me than "((positive + 1.9208) / (positive + negative) - 1.96 SQRT((positive * negative) / (positive + negative) + 0.9604) / (positive + negative)) / (1 + 3.8416 / (positive + negative))*"

YMMV, but I'd rather use an elementary and mathematically sound statistical technique than macguyvering something myself. (Though I do understand that, as some people describe in this thread, there can be different purposes to ratings and hence a need for different sorting mechanisms.)

You're right that confidence intervals depend on both quality and quantity, but the reason for this is to account for uncertainty. As n goes up, the standard error goes down to practically 0, and so e.g. a 4.2/5 movie with 100 reviews is still likely to be sorted higher than a 4.1 movie with 200 reviews. Quantity only comes into play when there is very little information to go on, after that quality becomes the driving factor.

Using internet points this way is not mathematically sound.

You're using a highly biased sample, to begin with. The mathematics here start with the assumption that you have a random sample. You don't. The assumption is invalid; and there's no reason to believe this calculation is a good one compared with any other.

A particular type of person votes on particular things: it's commonly observed that new movies rate higher than they should on IMDB, because super-fans are the first to vote. (This is convenient for your example)

Inappropriate for obtaining unbiased confidence intervals, yes, but this doesn't matter for ranking if bias is uniform across everything that's being rated.

You make a good point that there might be differential bias depending on when the movie came out, but I don't think the solution is then to say "well, now all bets are off, might as well concoct our own techniques and assume they're just as good or better." Statistical techniques are not either fully valid or fully invalid. Simulate the bias and look at exactly how it influences the results of a particular technique, and then perhaps use that to make an adjustment based on data rather than intuition.

It's not about macguyvering something, it's about reflecting on how well different solutions stack up with what you actually want to do. There is no one-size-fits-all ranking formula, at least in my opinion.

I’ve also seen a variation of this that gives different weights to positive and negative votes, the theory being that the likelyhood of you actually voting when you like/don’t like isn’t the same.

This is true, especially for things like app store ratings - people are much more likely to leave a rating if they really don't like something. Or if they really do like something for that matter.

Nowadays some apps encourage users more to leave a rating though. I was under the impression that Apple did not approve of those practices, although I might be confused with another rule that stated apps should not suggest 5-star ratings.

The Play Store, iirc, and probably Steam too, have a smart algorithm that display the 'most helpful' reviews at the top, so that as a potential customer you can determine the pros and cons of the app according to other users. Ratings are often very polarized - like up / downvotes.

Absolutely agree. The UrbanDictionary ordering seems nearly perfect for most internet purposes.

If this article wants to make its point, it should show cases where its ordering differs from UrbanDictionary.