Ultimately, the problem is people aren't actually benchmarking any of these ideas or if they are its via an extremely misleading methodology. For example, headroom violates cache frequently, nullifying any potential savings it could have; but does their benchmarks show that? Nope. [1]
As an aside, I wonder how many days are we away from Codex or Claude taking all the "read" tool calls and re-assembling a full map of your codebase and then offering it as a feature.
I have more faith in companies with a more targeted approach. Eg gzip does fine, but video codecs beat compressing raw video by a ton.
> As an aside, I wonder how many days are we away from Codex or Claude
That sounds like SourceGraph but twice as expensive, although it does have “AI” so probably lol
Eg absurd idea, but you could write something that minifies a codebase (by token, rather than byte) and then translates edits back into the expanded code. Probably an insane use of fuse lol. Partially minifying on each tool call sounds like a huge pain with a lot of state to track.
There’s also a lot of common situations where humans prefer solutions that take more tokens because it’s easier for us to read (eg for loop vs map vs list comprehension), which may have some gains.
I strongly suspect there is some form of token compression that works, but I don’t think it will be as simple as “pipe arbitrary text with no context into this tool”.
Jetbrains feels like a place this might come from. “Take this code, parse it to an AST, find the fewest token representation of it” feels like something they’d do, or maybe Astral (specifically in Python land, type checkers feel sort of adjacent as well).