Transformers are Turing-Complete, but the claim that LLMs are is a stretch. A transformer can replicate any function, same with other neural network architectures, but the question remains whether or not a transformer can be trained to generate natural language as well as be Turning-Complete.

Most natural languages are not Turing-Complete and lack formal rules making it an open question if a model trained on natural languages and formal languages can be deemed as Turing-Complete and so far that has not been proven.