Some studies are trying to ensure that the model reasons through abstractions instead of linguistic representations. (Of course the phenomenon of reasoning in substantially different quality depending on input language signals a fault - reasoning is beyond "spoken" language.)

In the past hours a related, seemingly important article appeared - see https://www.quantamagazine.org/to-make-language-models-work-...