News for hackers. Stuff that matters.
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by jeremyjh on Tuesday July 28, 2026 @12:41PM (#49086534)
It can easily be both. Also, they didn't use this innovation in K3 - K3 pre-training would have started months ago and the paper only mentions a 48B model. The people working at this level may not even be heavily involved in shipping a new iteration of K3, or at least theory contributions to it were done many months or even a year ago and after that it is all engineering.
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by vikramkr on Tuesday July 28, 2026 @01:19PM (#49087061)
This paper is from last year