did you not read https://arxiv.org/pdf/2410.05229

"We hypothesize that this decline is due to the fact that current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data."

They've presented a hypothesis warranting further study.

Here is an article where a model's learned ability to carry out modular addition is recovered. https://arxiv.org/abs/2301.05217

As you can see, the jury's still out.

Didn't Godel demonstrate that language can only approximate reality?

LLMs are a special type of transformer model, however several recent models have been multimodal, allowing for a variety of input and output type. So the question is if a neural network can approximate reality, not LLMs.

Though really, all embedded models of reality are approximate by nature, and based on stochastic empirical data. The real tradeoff with probability-based neural networks is whether a rigid algorithmic approach or a more flexible, but stochastic, approach solves the problem at hand.