News for hackers. Stuff that matters.
Re:Ggml.ai joins Hugging Face to ensure the long-term progre... by zargon on Saturday February 21, 2026 @10:01AM (#47101456)
Total parameters, not active parameters, is the property that matters for model robustness under extreme quantization.

Once you're swapping from disk, the performance will be quite unusable for most people. And for local inference, KV cache is the worst possible choice to put on disk.

There are no comments on this comment yet.