Need I remind you of the AI winter?
Progress isn't linear. You want to believe progress will continue at its current pace. I'd suggest it's every bit as reasonable to assume progress will stall out as current techniques plateau, just as they did in the past.
But there's no reason to believe that this technique will plateau before some noteworthy results are obtained.
Other than being connected by the term 'AI' what makes you believe that the past performance of researchers is indicative of future performance?
Are any of the limitations of the Apollo program inherent limitations of space exploration? How are they connected?
Absolutely there is, and the article specifically touches on it.
Because these models do not encode semantics, there's very little reason to believe the hallucination problem can be solved with current techniques.
In fact, OpenAI themselves have been downplaying expectations about GPT-4, and my suspicion is that's because these techniques are already starting to plateau.
I fully expect that GPT-4 will create even more natural-seeming language, with greater sentence variation, less repetition, etc, and with no change (or possibly even an increase) in the frequency of factual errors in the results.
Let alone other hyped technologies that have failed to deliver, like self-driving cars, voice assistants, AR/VR...
Heck, the AI winter was preceded by a boom in things like expert systems, and I'll bet folks back then thought it'd never end, too.
I'm willing to put my money where my mouth is. I'm willing to enter into a $$$ wager and assert that AI, specifically LLMs or similar systems, will only become more important and more in demand in 5 years.
I see a spectrum of responses to GPT, but the really important thing is the rate of change. Even if GPT somehow doesn't impress you currently, why do you think things won't be improved in 6 months? What about 5-10 years from now? I can understand thinking that transformers are maxing out on their sigmoid curve of progress, but every time people complain about perceptrons, RNNs, CNNS, etc - another team invents something new and blows out all the benchmarks. These things won't get worse, so even if you assume a pessimistic rate of change, how different will things look soon?
The observation that these AIs are "mere text generators" is a testament to how insane they are IMO. The fact that they're just predicting the next token and are still this powerful is paradigm-shifting for how I viewed human intelligence. I don't think these things are sentient, but if they can replace me at work in 10 years or less, who cares?