My point is, GPT-3 is operating at human levels for certain contexts. I think it would get passing grades on essays in a lot of schools in the US, for instance, just based on syntax and grammar.
Given what I've seen so far with GPT-3, that simple idea would have to have already been discussed at length on forums on the internet and in the corpus.
Usually books have facts and studies that they use as supporting points. Many of the connections they make between the subject material and their thesis are unique, and this forms their supporting argument. GPT-3 is rearranging words and sentences to resemble structures it's seen before, but it does not create novel facts.
The interesting part is that GPT-3's leap in performance can be attributed to scaling. That's easier to do than inventing completely new approaches. Scale data, scale compute, scale money, then you have something you couldn't have invented directly.
If you give me a huge corpus of chinese texts and a very long time, I might be able to figure out what character goes with what other, find the various structures in the text and then be able to generate a somewhat convincing made up chinese text while still not understanding a word of it.
These GPT-3 demos are impressive because they look like real text with proper syntax and grammar, but they still express absolutely nothing. It reads like a long series of rambling that goes nowhere. There's no intent behind it.
It reminds me of these videos of apes imitating humans by using their tools, banging hammers ineffectively. They are able to copy the appearance of the behavior, but not the reasoning behind it. They don't get why we bang hammers or what it achieves.