> We will see improvements but they will be very incremental, like GPT-3 to ChatGPT was barely noticeable and took years.
This was not my experience having tried GPT-3 before ChatGPT came out; and likewise people who have tried Bing Chat report it seems to be significantly more capable than ChatGPT.
Also, while it is clearly limited in competence, for breadth of information and speed and price it is clearly already superhuman. If that's useful or not remains to be seen, as the self driving car analogy is that the reaction times being effectively instant doesn't help when the perception says "what stationary truck?"
I kinda hope this won't be taking over for a decade, and you're not the only person who is suggesting we need multiple new breakthroughs on par with transformer models, but as the saying goes, predicting is hard, especially about the future.
Either way GPT-2 to GPT-3 was a much bigger step and happened much quicker. What we see now are already after much fine tuning, testing with humans, data filtering etc, more of that will result in smaller and smaller improvements. There is so much money spent on these models that they have already tried so many things, so getting to the edge of the S curve happens much quicker.
For the AI hype to pay off we would need all this buzz and activity to result in another new revolution similar to transformers. But without it it will go the same way as self driving cars, barely being good enough to do stuff, but with tons of works and checks it might lead to some automation at scale that could threaten jobs in a decade.