Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by korbip on Wednesday February 04, 2026 @11:25PM (#46895660)

This was done already here as well: https://arxiv.org/abs/2507.04239

[ Parent | on HN ]

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by cubefox on Thursday February 05, 2026 @09:45AM (#46900198)

Sounds interesting, but...

> these models dominate both exponential attention and linear attention at long-context training

There is no exponential attention; standard attention is quadratic. Strange mistake.

[ Parent | on HN ]