Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

Re:Ggml.ai joins Hugging Face to ensure the long-term progre... by jychang on Saturday February 21, 2026 @06:29AM (#47099787)

> How hard would it be to systematically evaluate the different quantizations? E.g. on the Aider benchmark that you used in the past?

Very hard. $$$

The benchmarks are not cheap to run. It'll cost a lot to run them for each quant of each model.

[ Parent | on HN ]

Re:Ggml.ai joins Hugging Face to ensure the long-term progre... by danielhanchen on Saturday February 21, 2026 @07:08AM (#47100030)

Yes sadly very expensive :( Maybe a select few quants could happen - we're still figuring out what is the most economical and most efficient way to benchmark!

[ Parent | on HN ]

Re:Ggml.ai joins Hugging Face to ensure the long-term progre... by illusive4080 on Saturday February 21, 2026 @07:55AM (#47100378)

Roughly how much does it cost to run one of the popular benchmarks? Are we talking $1,000, $10,000, or $100k?

[ Parent | on HN ]

Re:Ggml.ai joins Hugging Face to ensure the long-term progre... by danielhanchen on Sunday February 22, 2026 @04:58AM (#47109784)

Oh it's more time that's the issue - each benchmark takes 1-3 hours ish to run on 8 GPUs, so running on all quants per model release can be quite painful.

Assume AWS spot say $20/hr B200 for 8 GPUs, then $20 ish per quant, so assuming benchmark is on BF16, 8bit, 6, 5, 4, 3, 2 bits then 7 ish tests so $140 per model ish to $420 ish/hr. Time wise 7 hours to 1 day ish.

We could run them after a model release which might work as well.

This is also on 1 benchmark.

[ Parent | on HN ]