Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by logicchains on Wednesday February 04, 2026 @05:00PM (#46892438)

Multiplication has some properties like being cumulative. If we assume the sequence has any specific properties then we no longer have a general sequence model.

[ Parent | on HN ]

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by direwolf20 on Wednesday February 04, 2026 @06:45PM (#46893593)

I think you meant commutative.

Attention also has some specific properties.

And sometimes results are just unexpected. Did you know that anything a Turing machine can do in t tome steps, a different Turing machine can do in O(sqrt(t log t)) memory cells? https://news.ycombinator.com/item?id=44055347

[ Parent | on HN ]