they don't seem to have a theoretical upper limit. more data and more parameters seem to just keep making it more advanced. Even in ways that weren't predicted or understood. the difference between a language model that can explain a novel joke and one that can't is purely scale. So the thought is with enough scale, you eventually hit AGI