News for hackers. Stuff that matters.
Re:Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding by petu on Thursday November 05, 2026 @12:21AM (#50375517)
> and the remainder of VRAM is occupied by the KV cache and whatever experts still fit.

Those days engines try to optimize "whatever experts" part and can use remaining VRAM as cache, moving hot experts to GPU during inference (but less used experts keep executing on CPU, yea).

e.g. llama.cpp recently added flag for that: https://github.com/ggml-org/llama.cpp/pull/29887

Re:Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding by Tade0 on Thursday November 05, 2026 @04:57AM (#50377299)
I was not aware of that, so I downloaded the latest release (which includes but doesn't document this flag), but performance unfortunately tanked in my case.

Even If I set the fit target to a value which without the cache would leave that VRAM unused, I get about a third of my usual token rate.