you assume they just reproduces the training set, but when they get big enough they start to "understand" things, when the input is really big it actually can never "guess" the next word in the batch, so it has to "learn" concepts

If you specifically asked for the best moves, presumably it would recite moves that correlated with won games, not just common moves.

Not quite. That's just not a useful way to think about it because the way it works is much "foggier" than any earnest chess player you compare it against. It doesn't consider rules and legal plays, or strategy, in the way a chess.com player might. It composes each next play (and each analysis) based on a lot of game and non-game factors in a small, moving window of context. It can forget what it's doing, it can lose track of formal rules, it can slip from one pattern into another, it can start reciting chess poetry instead, etc

It's more like playing chess with someone suffering from dementia. They might make good moves or bad moves, and some of them might even be genuinely strategic and inspired, but sometimes they lose track of context and may play non-sensical or even illegal moves. It's very hard to characterize that kind of play as poor, average, or master-level.

If you play moves that are common across low-rating games, the average completions will be low-level moves. If you play moves that are common in high-rated moves, it should play high rated moves.

In practice, it mostly plays gibberish that doesn’t follow the rules.