News for hackers. Stuff that matters.
Re:Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA by mlazos on Tuesday July 21, 2026 @07:44PM (#48999913)
How can you be hitting cache on what I think are novel LLM prompts …
Re:Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA by oceansweep on Tuesday July 21, 2026 @07:59PM (#49000047)
Not them but my understanding is that the harness will send a simple 'heartbeat' message to keep the cache 'warm', (see prefix caching: https://handbook.modular.com/inference-optimization/prefix-c... ) which can then be edited/changed, which does cause the user to incur a fee, but its much less than the amount they'd pay on a no-cache hit request.
Re:Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA by verdverm on Tuesday July 21, 2026 @08:02PM (#49000063)
multi turn sessions, they are typically in the high 90% hit rate across all providers without doing much of anything
Re:Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA by slopinthebag on Tuesday July 21, 2026 @11:08PM (#49001366)
The prompt is only novel the first time it’s sent. Then as it’s sent repeatedly as part of the previous context it’s cached.

Deepseek afaik has a novel architecture that is somewhat forgiving of cache shifts. I’ve been getting +90% cache hits with Zed’s agent and it’s not doing anything special regarding caching afaik.