I meant it in the sense of: given any reasonable definition of "understanding", finding a mapping between the sequences of words and the structure of the world must satisfy the definition.
>I think it is a fairly common intuition that one cannot understand something just by rote-learning a bunch of facts.
I agree, but its important to understand why this is. The issue is that learning an assignment between some words and some objects misses the underlying structure that is critical to understanding. For example, one can point out the names of birds but know nothing about them. It is once you can also point out details about their anatomy, how they interact with their environment, find food, etc and then do some basic reasoning using these bird facts that we might say you understand a lot about birds.
The assumption underlying the power of these language models is that a large enough text corpus will contain all these bird facts, perhaps indirectly through being deployed in conversation. If it can learn all these details, deploy them correctly within context, and even do rudimentary reasoning using such facts (there are examples of GPT-3 doing this), then it is reasonable to say that the language model captures understanding to some degree.
This seems a bit tautological to me: if being able to make certain mappings is understanding, then does this not amount to "once we understand something, understanding it is a solved problem?"
On the other hand, the apparently simplistic mappings used by these language models have achieved way more than I would have thought, so I am somewhat primed to accept that understanding turns out to be no more mysterious than qualia.
I doubt that just any mapping will do. One aspect of human understanding that still seems to be difficult for these models is reasoning about causality and motives.
I think it is a fairly common intuition that one cannot understand something just by rote-learning a bunch of facts.