Re:Attention at Constant Cost per Token via Symmetry-Aware T...
by fheinsen on Wednesday February 04, 2026 @01:47PM
(#46889911)
Yes, there must be a connection. While adaptive truncation may prove impractical, it should be possible to measure spectral statistics on sample data, and specify a different fixed truncation order per layer, per head, etc. The github repository lists many other possible improvements: https://github.com/glassroom/sata_attention#proof-of-concept
[ Parent
| on HN ]
There are no comments on this comment yet.