Re:Generative AI and the big buzz about small language models
by sal9000 on Friday March 01, 2024 @07:28PM
(#39568604)
It came earlier than Mamba. It uses hyena hierarchy blocks, which are considered SSM but not the same as Mamba.
Re:Generative AI and the big buzz about small language models
by milliondreams on Monday March 04, 2024 @02:15AM
(#39587814)
We covered state space models in a blog post here - https://blog.dragonscale.ai/state-space-models/
It gives overview of Mamba And StrypedHyna.
[1]: https://arxiv.org/abs/2312.00752