TLDR, as the article is paywalled:

* You have to win a bunch of games against grandmasters, earning 'GM Norms', in order to become a grandmaster yourself.

* It is normal, at some contests, for lower-ranked players to pay to attend, while GMs are paid for attending.

* It is widely rumoured some contests are "norm factories" where fee-paying players get matched up with washed-up GMs who essentially throw the game.

* There are even rumours of contests where an aspiring grandmaster's parents offered cash to their opponents. There are even rumours of contests that took place only on paper.

yeah, sadly there are some dirty things going on when it comes to Norms. I guess with a better leadership and a clear policy of not tolerating pre-arranged games, this should change. But as with doping, it will never be erased 100%.

Pay money to advance in ratings? Nonsense! Just come up with a clever way to employ Stockfish over the board!

After reading that article it seems like it would be better to replace the “grandmaster norm” part of the process with standardized tournaments against a calibrated computer engine.

While it might solve issues with the norm process, it introduces more problems, specifically that two players of equal rating can have vastly different performance against a computer because they don't play like humans do.

Why is that a problem?

Because the purpose is to establish how good people are at playing other people, not engines.

While computer engines that are significantly better than the best chess players are plentiful, computer engines that are human rated and play human like moves do not exist. From the definition of ELO you could make a 1500 rated chess engine that plays perfectly half of its games and plays random moves the other half of the games, and it would stay at 1500 ELO if it continued to play against 1500 ELO players. This is an extreme example but it is not too far off from what chess engines do to reduce their ELO.

> computer engines that are human rated and play human like moves do not exist

I submit that you're wrong there: https://maiachess.com/

I have played a few hundred games against the various levels of Maia (https://lichess.org/@/maia1), and it has helped me improve from 1100 to 1425 blitz rating on lichess (along with playing against stronger players in the arena tournaments). It does seem to encapsulate the “average” playing style and common mistakes of the different ratings. It feels much fairer, more instructive, and more relaxing than playing against Stockfish, who will wildly blunder and then subsequently torture you.

However, even as a weak player, Maia is exploitable in a way that even novice humans are not. For example, it loves to give up obvious back-rank mates and almost never protects against it (e.g. with h3). It is easy for humans to spot if they are in danger of back-rank mate (especially if you’ve been burned before) and defend against it, so it does not happen much even at low levels. But Maia, game after game, gifts you an obvious mate in 1.

> computer engines that are human rated and play human like moves do not exist

Without knowing much about chess, I'd be inclined to say this can't possibly be true. Predicting human chess moves given an ELO rating sounds like a much easier task than training it to be good at the game. It is a different task alltogether, but until you reach relatively high elo scores, I'd expect a fake human ai would have a much easier time simply because it only needs to plan a few moves ahead to mimic the mindset of a lower level player.

The problem is that humans don't play that way, by evaluating a tree a few moves ahead and selecting from that. Humans use conscious heuristics and unconscious intuitions to make their moves that would be hard for a computer to mimic. It's easy to make a machine complete a task better than a human would, much harder to make it convincingly human-like in its behavior. Consider the task of picking apples from a tree. You could easily make a machine to do that. But what if you had to make a machine that to an onlooker (who couldn't see the robot's "face", let's say) would appear to be a human picking apples from a tree? And these people aren't just glancing at the apple-picking robot, they're spending their entire lives painstakingly analyzing the movements of this machine and trying their hardest to predict how they will move next. And the people doing this are self-selected to be the best performers in the world at predicting the moves of this apple-picking robot. Think you can make the apple-picking robot that will fool these people??

There are many chess AIs on chess.com specifically designed to play "like" a specific grandmaster or well known chess streamer. I don't think any titled player would not be able to guess they're playing a computer if they played a few games against the AI without being told. It's very well known that computer moves are very different from human moves, even the ones specifically designed to represent a human.

> The problem is that humans don't play that way, by evaluating a tree a few moves ahead and selecting from that.

As a modest 1400 blitz player, I definitely evaluate future states. That's not all I do, but it's certainly one thing I do. AlphaZero can also run just intuiting the next move without evaluating any additional states also fwiw. Though it is much better when it is allowed to do so.

> Consider the task of picking apples from a tree. You could easily make a machine to do that. But what if you had to make a machine that to an onlooker (who couldn't see the robot's "face", let's say) would appear to be a human picking apples from a tree? That would be much harder.

This is a very deep and nuanced task, made doubly difficult by obscure robotic hardware requirements. Not a great parallel.

> There are many chess AIs on chess.com, specifically designed to play "like" a specific grandmaster or well known chess streamer. I don't think any titled player would not be able to guess they're playing a computer if they played a few games against the AI without being told. It's very well known that computer moves are very different from human moves.

Maybe? A quick google suggested Maia is a close match to what I was suggesting. People are suggesting it does feel like a human in the thread. https://www.reddit.com/r/chess/comments/k4o6z1/introducing_m...

I don’t know that human calculation is so fundamentally different than computer evaluation, it’s just that it’s much, much slower. The human GM’s eval of any given position, without thinking ahead at all, is probably better than most engines. The problem is that strong evaluation isn’t worth much compared to a weaker evaluation that is nevertheless several candidate moves wider and many plies deeper (you might argue AlphaZero disproves this, but Stockfish was regularly beating the neural network engines even before its own neural network, for example).

Humans see fewer candidate moves. They regularly miss quiet moves that are the strongest. They often calculate to a certain depth (even a very shallow one) and stop based on gut instinct. But it’s still just trees of moves with some eval function.

I don’t think it’s fundamentally that hard to mimic, and it would actually be genuinely interesting for didactic purposes. But for fairly obvious reasons it’s not a priority outside of a couple of projects.

I think the proposal is to train neural networks on games played by human players at a given ELO.

Even if it were possible, I don't think there's a way to do it in a way that would make any sense. Would we set up an AI now and then forever use the same one? In that case people will learn any small quirks of the AI and optimize against those vs human play, fundamentally changing the game. If the proposal is to be constantly updating this AI, well even in that case I'd argue that people will over time identify any small difference between machine and human play, but in that case people will also be upset that someone got normed against the "easy" AI, or against the AI that didn't know about the dumdenmorph-joyce-allens countergambit yet, etc.

That kind of engine exists! It's called Maia, as mentioned in a sibling comment.

https://maiachess.com/

I haven’t followed the new wave of DL based engines but in the old Alpha-Beta algorithm days engines could be tuned by giving them a fixed processing budget per move (or a more sophisticated processing budget to allow for bursting).

Is that not a viable strategy on the new engines?

I think it's really hard to make it play like a human. Even GMs miss two-move tactics sometimes, wheras computers are pretty perfect at this.

On the other hand traditional algorithms might not be very good at positional chess when on a budget, wheras even weak human players can understand basic concepts like putting your rook on an open file. Then it gets even more complicated with the NNUE position evaluation models - maybe even with a one-move budget the neural net is still implicitly strong at understanding several moves ahead, and could still beat a strong human player.

Yeah exactly - it's easier to build an engine that plays as well as possible compared to an engine that plays reasonable/imperfect moves like a strong human player.

I believe this has not been true for some months now.

Computers are far stronger than humans and win every game. They cannot be artificially weakened as they do not play like weaker humans, but rather make a mixture of incredibly strong moves and random mistakes. As such, it would be a tournament of trying to lose the best game against a computer, which is not an accurate simulation of what human-human chess is like.

This isn’t really how most engines work. Reducing the depth that engines calculate to, or altering how they weight certain aspects of evaluation, are both perfectly natural ways to reduce an engine’s strength. They still won’t play like a human, but they don’t need to be forced into random mistakes.

Your experience with engines then is very different from my own and that of the wider chess community. The general consensus is that playing computers at any strength is both counterproductive and unfulfilling. They do indeed make random blunders if you lower their search depth. The engine will play brilliant moves for any tactic that falls within the search depth, but fail miserably for tactics that fall one ply outside of it. Humans function very differently to computers: they evaluate a fraction of the positions, but use a much more sophisticated evaluation function. There is no way of emulating human weakness with such a vastly different style of computation.

Here is a game I just played that illustrates this phenomenon:

https://lichess.org/PiIFqI2c/black

The engine plays well before making a series of random blunders no human would make. You can try this yourself by playing the different Stockfish levels on lichess.org. You will be unable to find a level that makes for an enjoyable game. There are some new engines that use neural networks to try and play similarly to how humans do [1]. I can't comment on their success, but their lack of wide adoption by the chess community signals to me that it is still incomparable to humans.

[1] https://maiachess.com/

I was answering the “they can’t be artificially weakened” point. I’m not saying they play like humans.

Ok, the implication in my comment was that they can't be weakened like humans, not that they cannot be weakened at all. The phenomenon of random mistakes does not need to be programmed in the engine, it is a byproduct of lowering the search depth or eval function.

Gotcha, apologies for the misread. I do think it’s an interesting problem space and worthy of more resources, there are all sorts of things that weaker, more human engines _could_ be helping humans with right now beyond traditionally just helping to memorise optimal lines. I don’t think this even means things like Maia that are perhaps human level but opaque, but rather being able to fit the kinds of heuristics humans use in eval or candidate move ordering. It would be great to be able to generate lines for specific opponent weaknesses (certain endgames, pawn structures, missed tactics etc). Even using engines to find things like forced draws from certain positions is harder than it should be right now. I’d love to see more chess _tooling_ rather than more pure engine research. If and when I retire from analytics in other sports this is what I’d like to spend my idle hours on.

Yup I agree, I think there's tremendous opportunity for computer tools to aid chess. Things like Lichess's recent ability to automatically categorize puzzles by the type of tactic was very cool, and only scratching the surface.

Sorry if I misunderstood but it sounds like you're saying engines aren't programmed to make random mistakes in order to lower playing strength, but this is how Stockfish works. Lowering the calculation depth and calculation time is used to reduce difficulty, but in addition to those it is programmed to make mistakes:

> Internally, MultiPV is enabled, and with a certain probability depending on the Skill Level a weaker move will be played.

Within the constraints of the depth and calculation times it determines the best move, but depending on MultiPV it will deliberately not play that move and play an inferior one instead.

This still doesn’t mean that engines can’t be made to play weaker moves without making deliberate mistakes, which is the only point I was making.

Maia plays like a human: https://maiachess.com

Why?

You can’t pay off a computer to throw a game.