News for hackers. Stuff that matters.

A comment on OK, I can partly explain the LLM chess weirdness now. Parent comment

Re:OK, I can partly explain the LLM chess weirdness now by int_19h on Friday November 22, 2024 @02:48PM (#42216820)
"LLM can reason" is trivially provable - all you need to do is give it a novel task (e.g. a logical puzzle) that requires reasoning, and observe it solving that puzzle.
[ Parent | on HN ]
Re:OK, I can partly explain the LLM chess weirdness now by staticman2 on Friday November 22, 2024 @04:16PM (#42217438)
How do you intend to show your task is novel?
[ Parent | on HN ]
Re:OK, I can partly explain the LLM chess weirdness now by int_19h on Monday November 25, 2024 @12:04AM (#42233387)
"Novel" here simply means that the exact sequence of moves that is the solution cannot possibly be in the training set (mutatis mutandis). You can easily write a program that generates these kinds of puzzles at random, and feed them to the model.
[ Parent | on HN ]