LLMs are statistical models. All it does is guess word sequences in response to prompts, like a 'roided out version of autocomplete. (This is why it hallucinates imaginary facts.) It has no ability to conceptualize or reason nor is there any credible proposal for a path forward to graft reasoning onto it.

The training data can be tweaked and more compute hours can be thrown at LLMs until it no longer makes financial sense to do so and then, as the OP said, it will hit a hard stop.