News for hackers. Stuff that matters.

A comment on Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation. Parent comment

Re:Attention at Constant Cost per Token via Symmetry-Aware T... by direwolf20 on Wednesday February 04, 2026 @06:52PM (#46893642)
Neural nets are structured as matrix multiplication, yet, they are universal approximators.
[ Parent | on HN ]
Re:Attention at Constant Cost per Token via Symmetry-Aware T... by noosphr on Wednesday February 04, 2026 @07:05PM (#46893778)
You're missing the non-linear activations.
[ Parent | on HN ]