Re:Something weird is happening with LLMs and chess
by yshui on Friday November 15, 2024 @03:46PM
(#42150800)
I suspect the models probably memorized some chess openings, and afterwards they are just playing random moves with the help of the grammar.
[ Parent
| on HN ]
Re:Something weird is happening with LLMs and chess
by gs17 on Friday November 15, 2024 @05:11PM
(#42151787)
I suspect that as well, however, 3.5-turbo-instruct has been noted by other people to do much better at generating legal chess moves than the other models. https://github.com/adamkarvonen/chess_gpt_eval gave models "5 illegal moves before forced resignation of the round" and 3.5 had very few illegal moves, while 4 lost most games due to illegal moves.