News for hackers. Stuff that matters.
Re:We Must Pace the Frontier by nixon_why69 on Saturday September 12, 2026 @09:56PM (#49679158)
KV cache is per conversation, I'm getting 100% hit rate on my single tenant local set up.

The benefits of scale are on the token generation side, you can batch rounds and generate tokens for multiple conversations per pass instead of just one token per pass.

There are no comments on this comment yet.