News for hackers. Stuff that matters.
Re:OpenAI Leaks 120B Open Model on Hugging Face by Nerd_Nest on Friday August 01, 2025 @01:11PM (#44759538)
Whoa, 120B? That’s huge.
Re:OpenAI Leaks 120B Open Model on Hugging Face by qeternity on Friday August 01, 2025 @01:29PM (#44759754)
120B MoE. The 20B is dense.

As far as dense models go, it’s larger than many but Mistral has released multiple 120B dense models, not to mention Llama3 405B.

Re:OpenAI Leaks 120B Open Model on Hugging Face by sciencesama on Friday August 01, 2025 @01:42PM (#44759916)
How much ram do you need to run this !!??
Re:OpenAI Leaks 120B Open Model on Hugging Face by cubefox on Friday August 01, 2025 @02:41PM (#44760725)
Probably about one byte per weight (parameter) plus a bit extra for the key-value cache (depends on the size of the context window).
Re:OpenAI Leaks 120B Open Model on Hugging Face by int_19h on Friday August 01, 2025 @03:32PM (#44761350)
You can go below one byte per parameter. 4-bit quantization is fairly popular. It does affect quality - for some models more so than others - but, generally speaking, a 4-bit quantized model is still going to do significantly better than an 8-bit model with 1/2 parameters.
Re:OpenAI Leaks 120B Open Model on Hugging Face by nivvis on Tuesday August 05, 2025 @02:14PM (#44801983)
for posterity, since shown that is it actually MoE

> 21B parameters with 3.6B active parameters