News for hackers. Stuff that matters.

A comment on Codestral: Mistral's Code Model. Parent comment

Re:Codestral: Mistral's Code Model by mauricio on Wednesday May 29, 2024 @11:24AM (#40512969)
22B params * 2 bytes (FP16) = 44GB just for the weights. Doesn't include KV cache and other things.

When the model gets quantized to say 4bit ints, it'll be 22B params * 0.5 bytes = 11GB for example.

[ Parent | on HN ]

There are no comments on this comment yet.