Re:Magistral — the first reasoning model by Mistral AI
by trebligdivad on Tuesday June 10, 2025 @09:40PM
(#44243392)
Nice! I'm running on CPU only, so it's interesting to compare - the Magistral-Small-2506_Q8_0.gguf runs at under 2 tokens/s on my 16 core, but your UD-IQ2_XXS gets about 5.5 tokens/s which is fast enough to be useful - but it does hallucinate a bit more and loop a little; but still actually pretty good for something so small.
[ Parent
| on HN ]
Re:Magistral — the first reasoning model by Mistral AI
by danielhanchen on Wednesday June 11, 2025 @05:01AM
(#44245616)
Oh nice! I normally suggest maybe Q4_K_XL to be on the safe side :)