If you sat someone down who had never played chess before, and taught them the rules, they also can't play chess. They wouldn't remember the allowed moves and would make tons of illegal moves.
The only way a person reaches the state of being able to play chess is to usually have learned the rules and also watched a bunch games. Either games between others, or most frequently games of "almost chess" that they play with a helpful instructor that corrects them every time they make an illegal move.
In that respect both humans and llms learn to play chess the same way - by watching games.
Or for an analogy, think about how many times you've sat down to learn a new board game and after the person who had played it before explains all the rules you still don't really get it. What's next? "Let's just play a practice round and I'll pick it up as we go." Ie learning from seeing the game played.
I think a lot of the skepticism in LLMs is people over estimate / inaccurately remember how people really learn / behave.
They can recite the rules perfectly, hence they remember them, I didn't say he just saw the rules once, both the LLM and the human has seen the rules enough to recite them perfectly.
Then a typical human can start to play chess based on those rules without having seen anyone play before, humans learn to play games by reading manuals all the time. You trying to argue against it here is absurd.
> think about how many times you've sat down to learn a new board game and after the person who had played it before explains all the rules you still don't really get it
I read the manual before playing board games because I like reading manuals, it is me teaching my friends then. You don't need someone to show you how to play games, reading the manual is enough...
Maybe you need two models to understand things, since the models wont understand itself but a model can understand the data encoded in the next model. This isn't an unfixable problem, but it is a problem that has to be solved.
But you wouldn't call that learning as a human. If you include a chess manual in the learning corpus of an LLM, but not any examples of playing chess, then the LLM can give you all the rules about chess, but it can't play chess. That isn't learning, that is just parroting.
A human who can recite the rules of chess but can't play chess, would you say that he understands the rules of chess? No, he just know how to output the words and not perform the actions, so there is no understanding there. Same with the LLM, nobody can say it understands these things since you would never said a human understood under similar conditions.