News for hackers. Stuff that matters.

A comment on Eagle 7B: Soaring past Transformers. Parent comment

Re:Eagle 7B: Soaring past Transformers by jeremycochoy on Tuesday January 30, 2024 @05:31PM (#39196839)
Well, RWKV is using some linear (non-quadratic) form of attention so... strictly speaking... you still need a bit of attention :D
[ Parent | on HN ]

There are no comments on this comment yet.