Trivially explained as "even a broken clock is right twice a day." I skimmed the paper, as it was linked here on HN iirc. First, it was published by Microsoft, a company that absolutely has a horse in this race (what were they supposed to say? "The AI bot our search engine uses is dumb?"). Second of all, I was very interested in their methodology, so I fully read the first section, which is woefully hand-wavy, a fact with which even the authors would agree:
> We acknowledge that this approach is somewhat subjective and informal, and that it may not satisfy the rigorous standards of scientific evaluation.
The paper, for instance, is amazed that GPT knows how to draw a unicorn in TikZ, but we already know it was trained on the Pile, which includes all Stack Exchange websites, which happens to include answers like this one[1]. So to make the argument that it's being creative, when the answer (or, more charitably, something extremely close to it) is literally in the training set, is just disingenuous.
[1] https://tex.stackexchange.com/questions/312199/i-need-a-tex-...
I don't think GPT-4 is magic, but unless the unicorn is literally an exact replica of something from it's training set, it clearly has "knowledge", and it's weird that you'd try to deny that.
Speaking of weird, that's a very weird definition of knowledge. Because in that case, then almost any data transformation operation implies knowledge. MS Word thesaurus? Knowledge. Search and replace? Knowledge. Markov chain[1]? Knowledge.
[1] http://www2.hawaii.edu/~chin/661/Projects/AdvAI_Project_Repo...
You can argue "it's only seemingly got knowledge!" all you want, as everyone else enjoys increasingly capable AI.
Trivial, vacuous, and wrong. It is not plausible to correctly predict the output of a serious Python program by coincidence. See the detailed examples in the paper -- one of which was pseudocode, not Python -- to see how silly this claim sounds.
> First, it was published by Microsoft, a company that absolutely has a horse in this race
Firstly, this is evidence that you are neither very familiar with Microsoft, nor with academic or industrial research labs. Microsoft Research (the affiliation of the authors) is practically a different company, and run more like a university lab. It is somewhat deeply insulting to their (often still primarily academic) researchers to suggest that they would publish a misleading puff piece to benefit the commercial arm.
Secondly, while the paper describes itself as qualitative, you can reproduce the major claims yourself (and I have).
The idea that Microsoft Research would publish anything remotely damaging to Microsoft is beyond naïve. I mean, one of their core tenets is "Ensure that Microsoft products have a future," but okay.
I expect that the reason people are constantly arguing with you is that your analysis does not explain some easily testable experiences, such as why GPT-4 has the ability to explain what some non-trivial and unique Python programs would output if they were run, despite GPT-4 not having access to a Python interpreter itself.