News for hackers. Stuff that matters.
Re:Mamba: The Easy Way by jackcook on Friday February 23, 2024 @04:04PM (#39486060)
Yes you're spot on, the nonlinearities come from the full Mamba blocks, which I left out of this post for simplicity/to focus on the bigger ideas the paper introduced. You can see it marked by the "X" on the right-most part of Figure 3 in the Mamba paper: https://arxiv.org/abs/2312.00752

There are no comments on this comment yet.