To add on, importantly, when Gebru called them stochastic parrots, CoT technique was not published yet, and LLMs had finite computational depth per emitted token, and that architectural limitation was a large contributor to hallucinations. But if you give a sufficiently general LLM file read/write (or arbitrary memory storage) and CoT before having to output a token, it is Turing-complete. Architecturally in principle you can have models that are accurate on studied domains, the question being how to grow a model that achieves this by reading literature and cheap RL on experiments, rather than curated design.