Re:Kimi Linear: An Expressive, Efficient Attention Architecture
by jeremyjh on Tuesday July 28, 2026 @12:41PM
(#49086534)
It can easily be both. Also, they didn't use this innovation in K3 - K3 pre-training would have started months ago and the paper only mentions a 48B model. The people working at this level may not even be heavily involved in shipping a new iteration of K3, or at least theory contributions to it were done many months or even a year ago and after that it is all engineering.
[ Parent
| on HN ]