Do Large Language Models learn world models or just surface statistics?

Re:Do Large Language Models learn world models or just surfa... by nickelpro on Sunday January 22, 2023 @09:47AM (#34477839)

If the only understanding of the rules is what has been tried/untried, that is definitionally a statistical model.

A valid rules model does not require to see all possible rules be played to understand the legality of a move.

So the LLM under study here could be at most, an invalid rules model. But an invalid rules model is indistinguishable from a statistical model, and the authors give no evidence for why we should believe it to be the former instead of the latter.

[ Parent | on HN ]

Re:Do Large Language Models learn world models or just surfa... by pyinstallwoes on Sunday January 22, 2023 @09:58AM (#34477953)

So, quantum mechanics?

[ Parent | on HN ]

Re:Do Large Language Models learn world models or just surfa... by karmakaze on Sunday January 22, 2023 @03:03PM (#34481646)

An interesting comparison is between AlphaZero and MuZero[0], the latter having not been given the rules and has improved upon AlphaZero in Go.

> "MuZero really is discovering for itself how to build a model and understand it just from first principles." — David Silver, DeepMind

So yes to both: statistical and a world model.

[0] https://en.wikipedia.org/wiki/MuZero

[ Parent | on HN ]