Why do you think that systems like GPT-3 are approaching an asymptote? Most people I've talked to say the opposite; they wouldn't have expected GPT-3 could be so much better than GPT-2 with no major additional breakthroughs.

It's just structurally wrong for the domain.

For instance, understanding language requires some of the capabilities of a SAT solver. This was something everybody believed in 1972, but today is denied.

Fundamentally "understanding" problems require the ability to consider multiple alternative interpretations of a situation, often choose one or work with the incomplete knowledge you have.

Back in the 1970s we had intellectually honest people like Richard Dreyfus writing books like "Things Computers Can't Do" that describe many specific ways the architecture at the time fall short. People on GPT-3 are working in a way that is academically valid (able to make results that are meaningful to a community) but from engineering it is like building a bridge with one end or a tall tower that carries no load.

GPT-3 has a structural mismatch with the domain it works in. Unlike early medical diagnosis systems like MYCIN, it is never a doctor, it just plays one on TV and it does the "passing for neurotypical" terrifyingly well.

The secret of GPT-3 is that people want to believe in it. Somebody will have it generate 100 text snippets and they will show you the three best. Your mind makes up meaning to fill up for its mindlessness. When this was going on with ELIZA in 1965 people quickly understood that ELIZA was hijacking our instinct to make meaning.

For some reason people don't seem to have that insight today, and it bothers me why that is. Back in the 1980s they had a lot of fear about compressing medical images because it could lead to a wrong diagnosis. Today you see articles in the press that are completely unquestioning that a neural network that has been trained to hallucinate healthy and cancerous tissues will always hallucinate the right thing when you are looking at a patient.

> People on GPT-3 are working in a way that is academically valid (able to make results that are meaningful to a community) but from engineering it is like building a bridge with one end or a tall tower that carries no load.

To me it seemed like the opposite. They are essentially working without any hypothesis of how their model actually works, without any model of the way it actually learns or the way it produces the results that it does, and instead placing blind trust in various metrics that are improving.

They are treating this as an engineering problem - how can we make the best human-sounding text generator - and not like a traditional research problem. GPT-3 has not taught us anything about anything except "how to generate text that seems human-like to humans". We have no firm definition of what that means, we have no idea of why it works, we have no idea of any systematic failures in its model, we know next to nothing about it, other than its results on some metrics.

Imagine the same applied to physics - if instead of inventing QM and Relativity or Mechanics, physicists got it in their head to try to feed raw data into a black box and see how well it predicts some observed movements.

In fact, this would be a pretty interesting experiment: how large would a deep learning model that could accurately predict what mechanics predicts get, given only raw data (object positions, velocities, masses, colors, surface roughness, shape, taste etc.)? Unfortunately, I don't think anyone has been interested in this type of experiment, because it is not useful from an engineering (or profit) perspective.

>Imagine the same applied to physics - if instead of inventing QM and Relativity or Mechanics, physicists got it in their head to try to feed raw data into a black box and see how well it predicts some observed movements.

In fact, this would be a pretty interesting experiment: how large would a deep learning model that could accurately predict what mechanics predicts get, given only raw data (object positions, velocities, masses, colors, surface roughness, shape, taste etc.)? Unfortunately, I don't think anyone has been interested in this type of experiment, because it is not useful from an engineering (or profit) perspective.

Isn't that what googles alphafold is doing pretty much?

https://deepmind.com/blog/article/AlphaFold-Using-AI-for-sci...

and it seems GPT-3 formed concepts related words together without being asked, its not picking the next best word strictly as a matter of statistic probability. So why wouldn't that apply to physics simulations / chemistry etc?

feed it chemical formulas and balancing equations from old chem 101 textbooks and it will fill in the blanks and start teaching itself how those things relate just by being corrected enough, then you can see if it has any predictive value.

I think both of your points are solving different problems that what I was suggesting.

My point is that an interesting scientific question is: "is the huge size of the GPT-3 model intrinsic to the problem of NLP, or is it an artifact of our current algorithms?"

One way to answer that is to apply the same algorithms and methods to mechanics data generated from, let's say, classical mechanics; and compare the generated model size with the size of the classical mechanics description. If the model ends up needing roughly the same amount of parameters as classical mechanics, then that would be a strong suggestion that NLP may intrinsically require a huge model as well. Otherwise, it would leave open the hope that and understanding can be modeled with fewer parameters than GPT-3 requires.

Your examples are still in this realm of engineering - trying to apply the black box model to see what we can get, instead of studying the model itself to try to understand it and how it maps to the problem it's trying to solve.

I think most people are still aware of the problem. I've seen a few people go off the rails, but most people have been well aware that "GPT-3 is having a conversation with me!" isn't itself particularly interesting.

On the other hand, we should acknowledge that humans are also structurally wrong for most of the domains we work in. A general-purpose neural network isn't a great tool to diagnose cancer, certainly - but it doesn't have to be great to exceed some radiologist's general-purpose light detectors. I think GPT-3 starts to edge into the territory of demonstrating Dreyfus was substantially wrong, and recognizably computer-like architectures are fully capable of doing abstract reasoning.

(That's not to knock on Dreyfus! Other voices in his era were optimistic to an absurd degree, and "come on guys our computers aren't that smart" was a very necessary response.)

> and crushing it into a high-dimensional vector in a continuous euclidean space. That's like fitting the 3-d spherical earth onto a 2-d map; any way you do it you do violence to the map.

Adding to this, most metrics can't be embedded well in euclidean space. Even something as simple as 4 nodes in a loop using the shortest path as your metric -- there's a minimum amount of error for any embedding into any euclidean space, and it's well above 0.

It's a bit surprising to me that we've hobbled along this far shoving square pegs into round holes wrt NLP since fundamentally that can't be fixed with more parameters and bigger coprocessors. It seems that some interesting features of natural language are actually euclidean.

> Adding to this, most metrics can't be embedded well in euclidean space. Even something as simple as 4 nodes in a loop using the shortest path as your metric -- there's a minimum amount of error for any embedding into any euclidean space, and it's well above 0.

Are you talking about a 2d Euclidean space, or about any number of dimensions?

Any number of dimensions actually :)

For that example it suffices to show it in 3D since any euclidean embedding of n+1 points can be isometrically embedded in an n-dimensional space, so if a 3D embedding with error E doesn't exist then neither does an ND embedding for any N>3.

Finding the minimum error requires a tad more effort, but it's not too bad to show that no embedding has 0 error:

Take a cycle a->b->c->d->a where every edge has length 1. Suppose a 0-error embedding exists. Points (a), (b), and (c) must be embedded colinearly, and then the only possible location for point (d) satisfying the distance requirements |a-d|=|c-d|=1 is precisely wherever we placed point (b), but then point (d) can't possibly have distance 2 from point (b).

By itself that doesn't show that infinitely small errors are impossible, but that assertion is also true in practice.

Thanks!

He's talking about curvature.

no, global topology, which is related to curvature.

For instance, I can go from Sydney to Sao Paulo going either East or West on the territory, but on the 2-d map you can only draw one path. You can map one point on the territory to multiple points on the map but that is itself a mismatch with the territory.

A model like WordNet, for instance, loses information about out-of-dictionary words. Words like "if", and "and" and "bit" are in the dictionary, maybe 95% of the words in your text are in the dictionary, but 50% of the meaning is in the out-of-dictionary words. There are things like FastText that do a little better (have a fighting chance of guessing at latin and greek words smushed together) but still make mistakes at an early phase of analysis which can't be recovered at later stages of analysis.

For a domain such as medical notes (say abstract of a medical case study) you might want to answer some question like "Did the patient die?" or "What code would I bill insurance for this?" and much more than half the time an embedding throws out an piece of information which is essential to computing the right answer as opposed to guessing at the answer.