Re:Attention at Constant Cost per Token via Symmetry-Aware T...
by quotemstr on Wednesday February 04, 2026 @06:23PM
(#46893377)
You can't stuff O(N) bits in O(1) space, so any scheme that purports, in general to do constant-time inference on unbounded context is snake oil, like a perpetual motion machine. Every such scheme must decay somehow. All you can do is choose how it decays.
[ Parent
| on HN ]
Re:Attention at Constant Cost per Token via Symmetry-Aware T...
by polynomial on Wednesday February 04, 2026 @09:58PM
(#46895091)
Right, not to "defend" the paper's claims, but it seems to be more like tuning how the leaky bucket leaks, using lossy compression to try to preserve some measure of coherency? Seems to turn on the fixed size summary.