News for hackers. Stuff that matters.

A comment on Vision Mamba: Efficient Visual Representation Learning with Bidirectional SSM.

Re:Vision Mamba: Efficient Visual Representation Learning wi... by nharada on Thursday February 01, 2024 @11:35AM (#39217800)
Has anyone tried this? I’ve seen the same people hyping this up all over /r/ml with multiple posts a week, marketing material, etc. Is this a legit new technique or just cleverly marketed paper?
[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by neuronexmachina on Thursday February 01, 2024 @12:12PM (#39218333)
The peer reviews on Mamba's ICLR2024 are pretty interesting, especially with how split the reviewers are: https://openreview.net/forum?id=AL1fq05o7H
[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by intalentive on Thursday February 01, 2024 @12:06PM (#39218242)
It’s legit and has been honed and refined for the past few years by the same Stanford PhD students (now graduated and working at CMU & Princeton).

The latest iteration has promising results for language but no one has yet trained and released a big (7B+) model to see how it scales.

[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by az226 on Thursday February 01, 2024 @11:57AM (#39218113)
Kind of weird we haven’t seen Mistral 7B Mamba. Until that’s out it’s untested vaporware.
[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by jasonjmcghee on Thursday February 01, 2024 @12:36PM (#39218680)
This isn’t a fine tune. It’s not even a transformer like Qwen, Llama, GPT, Gemini, or Mistral. It’s another approach to language models based on RNNs
[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by Solvency on Thursday February 01, 2024 @01:23PM (#39219309)
Doesn't the point remain though -- when/where do we demo this?
[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by junipertea on Thursday February 01, 2024 @01:24PM (#39219341)
There is Mamba 7B pre trained model on hugging face/ LM studio right now.
[ Parent | on HN ]
Re:Vision Mamba: Efficient Visual Representation Learning wi... by neuronexmachina on Thursday February 01, 2024 @12:21PM (#39218457)
Do you mean include it with the Mixtral "Mixture of Experts" model? I'm not sure Mistral Mamba makes sense, since it's a completely different architecture.
[ Parent | on HN ]