Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by vaillant on Monday December 04, 2023 @04:48PM
(#38523611)
Well, this is cutting edge ML code written in PyTorch. I wouldn't worry about understanding something like this - you start with scikit-learn first.
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by junipertea on Monday December 04, 2023 @04:15PM
(#38523172)
It corresponds to the paper formulas as a courtesy. While descriptive comments or function signature might be nicer, it still is reasonably easy to follow along from the paper description! (Although, most of the complexity is hidden away in selective_scan_cuda.fwd())
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by triyambakam on Monday December 04, 2023 @04:19PM
(#38523245)
Aha, thanks, that is helpful. Any chance you know of a guide that walks through a paper and code side by side?
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by ttul on Tuesday December 05, 2023 @10:06AM
(#38531762)
Hailing Jeremy Howard and his team at Fast.ai to make it so!
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by telotortium on Monday December 04, 2023 @07:27PM
(#38525378)
Implementation of selective_scan_cuda: https://github.com/state-spaces/mamba/blob/6dbfc4553a98c81e1...
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by ekelsen on Monday December 04, 2023 @04:13PM
(#38523152)
The variable names A, B, C, D and related assume knowledge of the state space model / formulation where these are the common names of matrices in the state space equations.
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by triyambakam on Monday December 04, 2023 @04:20PM
(#38523250)
Thanks, that's helpful.
Re:Mamba: Linear-Time Sequence Modeling with Selective State...
by filterfiber on Monday December 04, 2023 @04:21PM
(#38523272)
> It makes me hesitant to leave Typescript and even Rust (for web tooling).
To be clear this is not what a "typical" python application/script looks like at all. While there's definitely an argument to be made about python's immature type support (especially compared to typescript/rust), don't use this as an example of python's readability.
This was written to closely follow the math equations, not to be a maintainable piece of software. It makes a lot more sense from a mathematics/academic perspective, but not from a software development perspective.
Strong typing support would do nothing for readability but make this harder to read. Typing support doesn't help your naming, commenting, formatting, etc. that makes this so hard to read.
[1] https://github.com/state-spaces/mamba/blob/main/mamba_ssm/op...