News for hackers. Stuff that matters.

A comment on Transformers Can Do Arithmetic with the Right Embeddings. Parent comment

Re:Transformers Can Do Arithmetic with the Right Embeddings by jayd16 on Tuesday May 28, 2024 @03:43PM (#40504779)
LLMs are deterministic. We just sample the results, no? Also, no reason AI needs not be deterministic.
[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by TeMPOraL on Tuesday May 28, 2024 @05:32PM (#40505852)
> LLMs are deterministic.

In theory, yes. In practice, parallelism combined with floating point math make current implementations fundamentally non-deterministic.

[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by Tycho on Tuesday May 28, 2024 @06:48PM (#40506415)
Can you elaborate more on the parallelism aspect?
[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by mensetmanusman on Tuesday May 28, 2024 @09:44PM (#40507456)
Not fundamentally, but extremely costly to keep track of everything. Still deterministic though.
[ Parent | on HN ]
Re:Transformers Can Do Arithmetic with the Right Embeddings by joquarky on Wednesday May 29, 2024 @03:55PM (#40516240)
Temperature cannot ever reach 0 (this causes a division error), so they are not deterministic.
[ Parent | on HN ]