Re:Do Large Language Models learn world models or just surfa...
by YeGoblynQueenne on Sunday January 22, 2023 @09:43AM
(#34477807)
>> They've been able to demonstrate that the trained Othello player has a model.
They haven't demonstrated that, it remains a conjecture. They show some
evidence that they claim supports their conjecture. This evidence comes from
"probing" (the classifier-based method you describe) which is a heuristic
method that lacks the power to demonstrate anything conclusively. This is as
the authors point out:
It’s believed that the higher accuracy these classifiers can get, the better
the activations have learned about these real-world properties, i.e., the
existence of these concepts in the model.
"It's believed" as in, we don't know, therefore we can't use to demonstrate.
They haven't demonstrated that, it remains a conjecture. They show some evidence that they claim supports their conjecture. This evidence comes from "probing" (the classifier-based method you describe) which is a heuristic method that lacks the power to demonstrate anything conclusively. This is as the authors point out:
It’s believed that the higher accuracy these classifiers can get, the better the activations have learned about these real-world properties, i.e., the existence of these concepts in the model.
"It's believed" as in, we don't know, therefore we can't use to demonstrate.