barazany

The variance people are seeing makes sense once you look at what is actually happening with token billing. Cache hits are discounted. Cache misses cost 1.25x. Your cache hit ratio is effectively your quota multiplier.

Two developers doing the same work can burn quota at wildly different rates depending on their session patterns. Someone who uses /clear after long breaks preserves the cache. Someone who resumes a 100K+ token conversation after an hour of idle time reprocesses everything from scratch at the penalty rate.

The source code has a feature called "willow" that tries to help with this. It fires when two conditions are met: 75 minutes idle and at least 100K tokens in the conversation. At that point it shows either a blocking dialog or a one-liner hint to /clear. The 75-minute threshold is not arbitrary -- the cache TTL is around one hour, so 75 minutes idle means the cache has definitely expired. Continuing means reprocessing the entire context at 1.25x.

Boris mentioned improvements are coming, and given the source it is clear what that means. Not server-side quota changes -- client-side cache hit ratio optimization. There are twelve distinct caching mechanisms in the codebase, each plugging a specific leak. Tightening any of them hands quota back to users without changing the limit itself.

Full analysis of the caching mechanisms from the source: https://barazany.dev/blog/claude-code-token-caching

I analyzed its compaction engine, 3-layer masterpiece of which I write in full here: https://barazany.dev/blog/claude-codes-compaction-engine

How do you make the setup? Which provider gives you Anthropic compatible endpoint with GLM ?

I recently helped debug a 600+ line Kusto query powering a production feature (years of data, many joins, summaries). My usual method is to break the query down and measure total CPU + scanned data for each step, but doing this manually quickly becomes unmanageable.

So I extended my Kusto MCP server to return usage stats, and wrote an MCP prompt that guides the analysis automatically — breaking down queries, collecting metrics, and outputting a performance report with bottlenecks + recommendations.

This led to finding the real issue (lots of tiny fragments of data building up daily), and allowed us to suggest merge policies and scoped joins that cut CPU and scanned data dramatically.