News for hackers. Stuff that matters.
Re:Llama3 implemented from scratch by inciampati on Monday May 20, 2024 @07:36AM (#40414372)
Quadratic vs linear is not an optimization. It's a completely new game. With selective SSMs (mamba) the win is that associative training can be run in sublinear time via a log-cost associative scan. So you go from something quadratic wrt input sequence length to something logarithmic. If that's just an optimization it's a huge one.
Re:Llama3 implemented from scratch by curious_cat_163 on Monday May 20, 2024 @09:59AM (#40415538)
Okay. Respect your point of view. I am curious, what applications do you think SSMs enable that a Transformer cannot? I have always seen it as a drop-in replacement (like for like) but maybe there is more to it.

Personally, I think going linear instead of quadratic for a core operation that a system needs to do is by definition an optimization.