How many people are using ChatGPT, Stable Diffusion, etc. for economically or personally valuable activities?
If (1) is true, then the answer to that question is "zero" or at least "close to zero". Do you really believe that?
If (2) is true, then it is also true to say that transformer models will never exceed today's capabilities by a significant amount at any time in the future. Do you really believe that?
The limitation is inherent in the core design. There is no overcoming. This is not a hurdle or a wall. It's a design flaw.
Is it totally useless to everyone? No. Not completely. It's like a coherent search engine: a way to find data that is close to other data. But "close to" in this case is only "semantically", and never "logically", so that's that.
Is it going to get any less useless than it is? Only slightly. "It" will never get better. The only better version of "it" is a completely new ground-up redesign that doesn't resemble "it" at all.
Language understanding doesn't magically spawn itself as a process on your computer! Someone has to write that program first.
And that's my point. ChatGPT transforms language, but it does not understand it. For that, we will need a different kind of program.
Do you think it's impossible for such a program to emerge as weights in a Turing complete neural network architecture?
What you are telling me is that I should place my expectations for the future, not on the reality in front of me, but on the hopes and dreams you have for the future. That's circular reasoning.
The very reason that I don't place credibility in your assertions is the lack of reason itself: in your assertions, and in what a neural network is.
Neural networks are like dreams. Wonderful only when your intention is to get lost in a swirl of memories. Useless if you want to actually accomplish something.
Knowing the difference is crucial, because that difference can never be taught to a neural network without completely redefining what a neural network is in the first place.
Knowing the difference is literally the thing neural networks are incapable of doing. They don't know anything. They just guess. That's literally the function. In the code. Guess what comes next.
There is no sense pretending sense itself will magically appear out of a guessing machine. Neural networks are nonsense generators, and that is what they are forever doomed to be.