LLMs trained on “A is B” fail to learn “B is A”

Re:LLMs trained on “A is B” fail to learn “B is A” by johnthewise on Saturday September 23, 2023 @08:56AM (#37622854)

Sure, that wouldn't be impressive if you wrote tree search for chess. However, if you wrote a program that doesn't do any search and still plays well, that'd be impressive. I don't understand your claim that LLMs playing chess is anything sort of understanding.

related: https://parrotchess.com/

[ Parent | on HN ]

Re:LLMs trained on “A is B” fail to learn “B is A” by xrisk on Saturday September 23, 2023 @09:27AM (#37623032)

How do you know there’s no search involved?

[ Parent | on HN ]

Re:LLMs trained on “A is B” fail to learn “B is A” by Jensson on Saturday September 23, 2023 @10:01AM (#37623301)

It does more illegal moves the more you deviate from standard chess openings and positions and becomes unable to play at some point. If it did a tree search based on chess rules that wouldn't happen. If it encoded a lot of chess openings and some tiny ad hoc logic around those then that is exactly what we would expect.

[ Parent | on HN ]

Re:LLMs trained on “A is B” fail to learn “B is A” by golol on Saturday September 23, 2023 @12:19PM (#37624615)

This is not true anymore. It has been discovered that gpt-3.5-instruct when prompeted with purely PGN notation.playes at >1700 Elo and makes no/little illegal moves.

[ Parent | on HN ]