https://en.wikipedia.org/wiki/Elo_rating_system#Practical_is...
Trueskill and many others are often used instead.
https://en.wikipedia.org/wiki/TrueSkill
The biggest issue is fun is not the same fair and its a lot more complicated to tune for fun.
You could, for example, play 100 matches where you were given the closest opponent but at a disadvantage for every single match. There's a lot of other scenarios around how you want to fold in ping based match making and how that smaller pool effects the fairness of the matchhing.
Elo tells you how skilled a player is, how surprising a result (win, loss, draw if applicable) is, and how a player's rating should change based on a result and how surprising the result was. It does NOT tell you who should be matched up against who.
> The biggest issue is fun is not the same fair and its a lot more complicated to tune for fun.
> You could, for example, play 100 matches where you were given the closest opponent but at a disadvantage for every single match. There's a lot of other scenarios around how you want to fold in ping based match making and how that smaller pool effects the fairness of the matchhing.
Elo tells the system that if you play 100 games at a disadvantage, your rating should not go down as much when you lose, and should jump up more on a win. Elo is the math that tells the system that if your skill does not change, your score does not change, regardless of whether you play 100 games at disadvantage and lose most of them, or play 100 games at an advantage and win most of them.
I agree that Elo handles uncertainty poorly; when you're rating a chess player whose career is tracked over the course of decades, this isn't an issue, but online play with transient accounts, players who blip into a game when it's hot and then never play it again, smurfs and account buyers, people who take a break for a while and come back, patches which might radically change the meta, etc, uncertainty should be accounted for. Note that TrueSkill is, for all intents and purposes, just Elo+Bayesian inference. Like Elo, it is a rating system, not a matchmaking system. TrueSkill does not handle the problem of playing 100 games in a row against players who are better than you. TrueSkill, like Elo, just tells you what a result means, not which players should be pitted against each other.
It's open-license, handles complex matchups (e.g. 5 vs. 3 vs. 4), comparable in predictive accuracy, and is up to 20x faster than TrueSkill. The implementation is also much simpler.
> You could, for example, play 100 matches where you were given the closest opponent but at a disadvantage for every single match.
not really, you'd derank aggressively and end up playing people at a lower level and presumably winning comfortably after some time.
It’s also a bad fit for team games. But ranking individual contribution to a team is hard. Even for something like the NBA where there’s obviously a ton of incentive to do this, statistical methods rely on a large sample size and often need to be taken with a grain of salt.