News for hackers. Stuff that matters.
Re:Do Large Language Models learn world models or just surfa... by nickelpro on Sunday January 22, 2023 @09:21AM (#34477632)
A human mind is not a rules model for Chess, and comparisons to a human mind are worthless. We're not asking if the LLM is human, only if it is modeling the game rules or modeling statistics

My point is this:

* If the results of the model were perfect, not a single counter-example, that would be a strong argument that the LLM has built an internal model of Othello

* If counter-examples exist, the result of the experiment is indistinguishable from a sufficiently clever statistical inference and proves nothing

Re:Do Large Language Models learn world models or just surfa... by hcks on Sunday January 22, 2023 @09:38AM (#34477769)
So you assert that parent doesn’t have an internal model of chess?

Or that only proved expert systems built specifically to play chess have an internal model of it?

Re:Do Large Language Models learn world models or just surfa... by nickelpro on Sunday January 22, 2023 @09:51AM (#34477880)
> So you assert that parent doesn’t have an internal model of chess?

I think it's an interesting question, I find it more likely we have an imperfect ability to perceive the whole chess board, whatever the hell that might mean.

In any case it's irrelevant to an LLM which has perfect information from the input set.

So perhaps we rephrase the question, I think the parent, if given a list of individual chess moves (in whatever notation pleases them), should be able to tell if a given move is valid or not for the given piece. If the parent failed to do that, I would say they do not have a valid model of chess, ya.

Re:Do Large Language Models learn world models or just surfa... by shawnz on Sunday January 22, 2023 @10:50AM (#34478520)
> If the parent failed to do that, I would say they do not have a valid model of chess

Surely even the world's foremost chess experts will occasionally make errors in such a situation, so what's the point of holding language models to an intelligence standard that even humans can't meet?

I see two ways of interpreting this: either language models do approximately learn world models, or else world models aren't necessary for human intelligence

EDIT: I see from your other posts in this thread you're probably more inclined to believe the latter statement, which wasn't obvious to me from your first post and I think that explains why there is so much contention in this thread.

Re:Do Large Language Models learn world models or just surfa... by noiv on Sunday January 22, 2023 @09:49AM (#34477856)
Well, if the model has a 100% grasp of the rules, why call it a model?
Re:Do Large Language Models learn world models or just surfa... by nickelpro on Sunday January 22, 2023 @09:58AM (#34477944)
Because it's not a physical othello board? It's just an internal model of an othello board encoded in the weights of a neural net?

Insert whatever semantics float your particular goat

Re:Do Large Language Models learn world models or just surfa... by noiv on Sunday January 22, 2023 @02:26PM (#34481322)
I thought the whole point was about modelling rules not atoms.