News for hackers. Stuff that matters.

A comment on Mamba Explained: The State Space Model Taking On Transformers. Parent comment

Re:Mamba Explained: The State Space Model Taking On Transfor... by p1esk on Sunday February 25, 2024 @06:20PM (#39505987)
I think they went MoE purely because straight up scaling from 175B to 1.8T is just too expensive. But it’s still 10x scaling, right?
[ Parent | on HN ]

There are no comments on this comment yet.