That's not an assumption that I am personally making. That's what experts in the field believe.
> "The Internet" basically encompasses all meaningful sources of data available, especially if we are talking specifically about software development. But even beyond that, it is very unclear what other high quality data it would consume that would improve the things.
How about, interacting with the world?
> I love how you conveniently sidestepped the part where I ask if it would improve the pervasive issue of generating plausibly sounding bullshit.
I was not trying to "conveniently sidestep". To me, that reads like a more emotional wording of the first question you asked, which is if LLM's are expected to improve. To that question I answered yes.
> The assumption that generative AI will improve is as valid as the assumption that it will plateau.
It is certainly not as valid to say that generative AI will plateau. This is comparable to saying that the odds of winning any bet are 50/50, because you either win or lose. Probabilities are a thing. And the probability that the trend will plateau is lower than not.
> It is quite possible that what we are seeing is "as good as it gets", and some major breakthrough, that may or may not happen on our lifetime, is needed.
It's also possible that dolphins are sentient aliens sent here to watch over us.
People invested in something believe that throwing money at it will make it better? Color me shocked.
"eat meat, says the butcher"
The rest of your answer amount to optimistic assumptions that yes, AI future is rosy, based on nothing but a desire that it will, because of course it will.
That makes an assumption that throwing dollars on AI training is a surefire way to solve the many shortcomings of LLMs. It is a very optimistic assumption.
> Why wouldn't there be? AI's are currently trained on the internet but that's obviously not the only source of data.
"The Internet" basically encompasses all meaningful sources of data available, especially if we are talking specifically about software development. But even beyond that, it is very unclear what other high quality data it would consume that would improve the things.
> The answer to this, is also yes. It is well established that, all else being equal, a bigger model is better than a smaller model, assuming that the smaller model hasn't already captured all of the available information.
I love how you conveniently sidestepped the part where I ask if it would improve the pervasive issue of generating plausibly sounding bullshit.
The assumption that generative AI will improve is as valid as the assumption that it will plateau. It is quite possible that what we are seeing is "as good as it gets", and some major breakthrough, that may or may not happen on our lifetime, is needed.