LLMs trained on “A is B” fail to learn “B is A”

Re:LLMs trained on “A is B” fail to learn “B is A” by Jensson on Saturday September 23, 2023 @10:01AM (#37623301)

It does more illegal moves the more you deviate from standard chess openings and positions and becomes unable to play at some point. If it did a tree search based on chess rules that wouldn't happen. If it encoded a lot of chess openings and some tiny ad hoc logic around those then that is exactly what we would expect.

[ Parent | on HN ]

Re:LLMs trained on “A is B” fail to learn “B is A” by golol on Saturday September 23, 2023 @12:19PM (#37624615)

This is not true anymore. It has been discovered that gpt-3.5-instruct when prompeted with purely PGN notation.playes at >1700 Elo and makes no/little illegal moves.

[ Parent | on HN ]