News for hackers. Stuff that matters.
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by thomasahle on Tuesday July 28, 2026 @11:11AM (#49085135)
Here's one way it could happen:

Let's say there's some circuit that does problem solving of the kind we call intelligence.

We dont know what this circuit looks like, but it exists in our brain.

Doing regression on outputs from the brain (e.g. internet text) with enough parameters, we can "fit" our model to this circuit.

But if you try to fit it with fewer parameters than it needs, you're just going to get some linear approximation.

Re:Kimi Linear: An Expressive, Efficient Attention Architecture by hendiatris on Tuesday July 28, 2026 @11:25AM (#49085350)
So basically a Nyquist rate type of concept.
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by esafak on Tuesday July 28, 2026 @11:44AM (#49085630)
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by lacunary on Tuesday July 28, 2026 @11:18AM (#49085249)
what is the approximation linear in?
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by lern_too_spel on Tuesday July 28, 2026 @11:27AM (#49085379)
In the output of this nonlinear model /s.