News for hackers. Stuff that matters.

A comment on Codestral: Mistral's Code Model.

Re:Codestral: Mistral's Code Model by isaacrolandson on Wednesday May 29, 2024 @03:23PM (#40515814)
Will this run on an M3 48GB?
[ Parent | on HN ]
Re:Codestral: Mistral's Code Model by piskov on Wednesday May 29, 2024 @03:48PM (#40516158)
You’ll need 44GB just for the weights

By default only 75% of unified memory is available to GPU if you have >36GB. So with 48 total only 36 is available for GPU with is lower than 44.

tldr; without quantization you will not be able to run it.

[ Parent | on HN ]