I think the next big leap will be the context size. Going from 4k to 32k tokens means that the model can see a lot more when generating the answer. For code especially this makes a big difference in the ability to reason about it.

As to the limits, it's hard to predict because what we're seeing is more of a series of sudden leaps. But let me put it this way - so far we've seen noticeable improvements on every iteration of the new LLMs. So given this, why should the default assumption be that the limit past which nothing new will emerge, whatever it is, is close to the present size of the publicly available models?