News for hackers. Stuff that matters.

A comment on Mistral 7B, the most powerful language model for its size to date, Apache 2.0. Parent comment

Re:Mistral 7B, the most powerful language model for its size... by olso on Wednesday September 27, 2023 @03:18PM (#37679817)
Hey, how did you try it on M1? I don't see any MPS support.

https://github.com/mistralai/mistral-src/issues/2

edit: nevermind https://ollama.ai

[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by anonyfox on Wednesday September 27, 2023 @03:19PM (#37679832)
`ollama run mistral` <-- literally thats it
[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by iamcreasy on Thursday September 28, 2023 @12:57AM (#37685396)
For anyone wondering - no need to download anything from Mistral website. Download ollama and use this command.
[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by KennyBlanken on Wednesday September 27, 2023 @08:41PM (#37683668)
Heads up that this app is extremely obnoxious and upon launch installs itself in to your startup items for zero fucking good reason.
[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by justinmayer on Thursday September 28, 2023 @02:49AM (#37686191)
Quite true. I actually submitted a pull request [1] over a month ago to make it clear to potential users what will happen when Ollama is launched for the first time, but based on the complete lack of response from Ollama developers, I get the distinct impression that they are reluctant to draw attention to these important details. Unfortunate.

[1]: https://github.com/jmorganca/ollama/pull/395

[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by jsnow119 on Saturday September 30, 2023 @04:58PM (#37719740)
How come one blog post with the torrent magnet of mistral says I need 24g vram but ollama says I only need 8g regular ram minimum?

If I wanted to experiment with working with models and finetuning offline should I just save for a 32g vram+ video card?

[ Parent | on HN ]