LLMs trained on “A is B” fail to learn “B is A”

Re:LLMs trained on “A is B” fail to learn “B is A” by johnthewise on Saturday September 23, 2023 @09:25AM (#37623026)

You are right, but evidence of LLMs working well with language is not a sign that they might be memorizing Chess. But rather opposite, infeasability of memorization of chess and yet LLMs still doing well on them might be a sign that LLMs could be doing the same 'thing' with language.

[ Parent | on HN ]

Re:LLMs trained on “A is B” fail to learn “B is A” by Jensson on Saturday September 23, 2023 @09:57AM (#37623255)

LLM are not doing well in chess. They do well when people play standard openings, but play against the LLM using an inexperienced player and the LLM will quickly start making illegal moves and be unable to play. There are many millions of pro chess games recorded on the internet, enough to cover all the openings and plenty of ways to go from there with some basic heuristics, but the more you diverge from those the more the LLM will break down since it doesn't understand the rules, it just memorized a lot of openings and some really primitive patterns to follow from there.

[ Parent | on HN ]

Re:LLMs trained on “A is B” fail to learn “B is A” by golol on Saturday September 23, 2023 @12:20PM (#37624632)

This is not true anymore, see this tweet.

https://twitter.com/GrantSlatton/status/1703913578036904431

I have tested it myself.

[ Parent | on HN ]