News for hackers. Stuff that matters.
Re:Attention at Constant Cost per Token via Symmetry-Aware T... by kristjansson on Wednesday February 04, 2026 @02:06PM (#46890179)
> approximately the same magnitude

and they really do mean that, their results show +/- 1 on log10 plots.

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by cptroot on Wednesday February 04, 2026 @07:50PM (#46894152)
I don't think this is an accurate characterization of the error magnitude? Their error plots (from appendix 3) are all showing `log_10(|Y - \dot{Y}|)` as having a median of ~-3 (difference of 0.001) and a max of ~1.5 (difference of 0.035), and this is with only 3 Taylor terms.
Re:Attention at Constant Cost per Token via Symmetry-Aware T... by kristjansson on Friday February 06, 2026 @01:01PM (#46916016)
Oh you're right that is a misread on my part, the appendix charts don't say that. I think they're just useless then though? Since they're reporting absolute error (on a log10 scale) we can't assess the relative to compare to the 'within an order of magnitude' claim in the text.