News for hackers. Stuff that matters.

A comment on Many options for running Mistral models in your terminal using LLM. Parent comment

Re:Many options for running Mistral models in your terminal ... by eurekin on Monday December 18, 2023 @03:48PM (#38688003)
I tried an hour ago and had a can't load model error. Everything up to date. Is there any special step?
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by mmcclure on Monday December 18, 2023 @03:53PM (#38688098)
Tried `ollama pull mixtral` just now and it seems to be working, albeit pretty slowly.
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by Casteil on Monday December 18, 2023 @04:20PM (#38688467)
How much RAM do you have? Mixtral is a beast and the non quantized model wants 40GB+ of memory.
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by eurekin on Monday December 18, 2023 @06:05PM (#38689652)
Ah, that might be it! I have only 32
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by ukuina on Tuesday December 19, 2023 @11:58PM (#38705605)
The q2 should fit.
[ Parent | on HN ]