You should check out the work referenced in the abstract, the most recent here: https://twitter.com/lambdaviking/status/1630581475425828864

There are limitations for what a Transformer can compute if we do not allow for Chain-of-Thought type of output. By allowing the model to "show its work" allows it to effectively use the output as an "infinite tape". I'm simplifying but that's the basic gist of it.

I'll shamelessly plug my blog post from a week ago for a simpler take on the matter: https://blog.wtf.sg/posts/2023-02-03-the-new-xor-problem/