News for hackers. Stuff that matters.

A comment on Transformers Can Do Arithmetic with the Right Embeddings. Parent comment

Re:Transformers Can Do Arithmetic with the Right Embeddings by jhanschoo on Tuesday May 28, 2024 @12:22PM (#40502420)
A basic transformer architecture performs only a bounded amount of computation per generated token, so it can never emulate a machine computing sufficiently hard problems.
[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by EVa5I7bHFq9mnYK on Tuesday May 28, 2024 @05:10PM (#40505640)
Yes, because it's feed forward. It must have loops to be a Turing machine.
[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by phkahler on Tuesday May 28, 2024 @05:51PM (#40505995)
It does. The output is fed back in.
[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by jhanschoo on Tuesday May 28, 2024 @09:59PM (#40507572)
It indeed does, but it must generate a token per loop, and can thereby solve some linearly complex problems, but it cannot solve harder problems.
[ Parent | on HN ]