News for hackers. Stuff that matters.
Re:Right-sizes LLM models to your system's RAM, CPU, and GPU by riidom on Monday March 02, 2026 @03:10PM (#47223404)
LM Studio has an option on model load that I believe does what you describing here: "K Cache Quantization Type" (and similar for "V"). It's marked as experimental and says the effect is basically hard to predict. Never tried myself, though.

There are no comments on this comment yet.