I think it's interesting to think about, and still somewhat uncertain:

* How much a large language model is effectively a world model (indeed, language tries to model the world...)?

* How much do humans use language in their modeling and reasoning about the world?

* How fit is language for this task, beyond the extent humans use it for?