I'm not sure that you can assert that language is tightly coupled to reality, unless you're using the term "reality" to mean something akin to "as one perceives the world" (regardless of whether that perception is correct or not).

Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and are subject to interpretation) they certainly weren't tightly coupled to reality in an objective sense.

Training on expressions of language (I separate the concept of language itself from its expression in the form of writing, speaking, etc) certainly has use cases but can GPT-3 recognize a previously unknown analogy and correlate it with the proper piece of applicable "knowledge" it has? If not then it really has no understanding.

a few things:

1. i get where you're coming from

2. yes, language is bottlenecked by human perception, as are all things

3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.

4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.

Why does it make human-like predictions as if it knows about reality? Because it's essentially pattern-matching the consensus of the literature it was trained on. Literature as a whole will tend to settle on a consensus sentiment, albeit one that is probably significantly behind the current consensus sentiment (it takes time to accumulate enough mass to move the weights). If your interactions with GPT-3 fall into the rather sizeable consensus that most people either subscribe to or are familiar with then it will certainly prove a decent mimic of understanding. If, however, you attempt to teach it a novel concept or if you dig into its interactions long enough to test for depth of understanding you run into the gaps and GPT-3 either begins to mimic that bullshit artist everyone knows that claims to know things but is only regurgitating platitudes and buzzwords or it begins to mimic behavior that would make you question whether it was sober and/or sane.

There's little doubt that GPT-3 could hold its own quite well in a bout of polite conversation and/or small talk that features in many social situations but that's more a commentary on the limited area of knowledge and behavior that etiquette expects for interactions in such settings.

It could also be trained on the canon of Shakespeare and behave as a prior work of Shakespeare, but if you left one play out of that canon and then attempted to have GPT-3 generate that missing work it wouldn't... at all. It doesn't approximate how Shakespeare thought, it approximates the literature he produced that you used for training.

None of this is to denigrate the achievement that GPT-3 represents, it is merely to point out that it is unfair to GPT-3 to attempt to hold it to the standards of AGI.