Re:OpenAI Leaks 120B Open Model on Hugging Face
by int_19h on Friday August 01, 2025 @03:32PM
(#44761350)
You can go below one byte per parameter. 4-bit quantization is fairly popular. It does affect quality - for some models more so than others - but, generally speaking, a 4-bit quantized model is still going to do significantly better than an 8-bit model with 1/2 parameters.
[ Parent
| on HN ]
There are no comments on this comment yet.