News for hackers. Stuff that matters.

A comment on Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation. Parent comment

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by cptroot on Wednesday February 04, 2026 @07:50PM (#46894152)
I don't think this is an accurate characterization of the error magnitude? Their error plots (from appendix 3) are all showing `log_10(|Y - \dot{Y}|)` as having a median of ~-3 (difference of 0.001) and a max of ~1.5 (difference of 0.035), and this is with only 3 Taylor terms.
[ Parent | on HN ]
Re:Attention at Constant Cost per Token via Symmetry-Aware T... by kristjansson on Friday February 06, 2026 @01:01PM (#46916016)
Oh you're right that is a misread on my part, the appendix charts don't say that. I think they're just useless then though? Since they're reporting absolute error (on a log10 scale) we can't assess the relative to compare to the 'within an order of magnitude' claim in the text.
[ Parent | on HN ]