Do Large Language Models learn world models or just surface statistics?

Re:Do Large Language Models learn world models or just surfa... by williamcotton on Sunday January 22, 2023 @10:02AM (#34477984)

> A model that "knows" the rules will make no errors, a model that makes any errors does not "know" the rules. Simple as.

Wittgenstein would ask, does it matter if the model “knows all the precise rules” or not?

Does it matter if I know all the precise rules? Can two of us have a fun game of chess if we basically know all the rules and mainly make legal moves or at least not notice when we break a rule or can agree on an action when we’re confused?

What’s important is that we or the model play the same game with others. We don’t need to worry too much about the definitions. We just need to observe the results to deduce if there was any sort of meaningful interactions.

What exactly is offsides in soccer? Or pass interference in football? Or a balk in baseball?

[ Parent | on HN ]

Re:Do Large Language Models learn world models or just surfa... by nickelpro on Sunday January 22, 2023 @10:07AM (#34478043)

If your claim is that you've produced a computer program that accurately models the rules, it matters a great deal. It is, perhaps, the only thing that matters.

A calculator that says 2+2=5, even 0.01% of the time, is not a very good calculator.

[ Parent | on HN ]

Re:Do Large Language Models learn world models or just surfa... by williamcotton on Sunday January 22, 2023 @10:14AM (#34478119)

Yeah, but we’re trying to play football here. We’ve already got calculators!

[ Parent | on HN ]