I burn a ton of tokens, but things actually have to prove their value. And the vast majority of things do not come close to doing so.
I have my own AI agent full of stuff. I blind A/B test everything, but I also don't think the results are all that useful as a signal to others.
Just because I Blind A/B test it 4 months ago, it's maybe not meaningful today.
Maybe the word choices I use dramatically impact things.
I do it, because I can prove the value, and see it with my own eyes. I don't even bother publishing the specific Blind A/B tests.
Also, I've seen other people try to Blind A/B test and get it very wrong. If your measurements aren't good, the test is meaningless.
I don't know. We're all working on these problems together. There's a lot of black magic (which is why I rely on hooks a lot). I'm sure I have tons of black magic, I have a large little AI Agent.
But what I know for certain, is it works for me. All it takes is for me to not use it, and I honestly don't know how everyone currently works with AI.
I will link it, but it is not an endorsement for what you do. Mostly only other software engineers use it. And it's so very specific to the things I have to do.
At best, maybe it sparks an idea for you to implement on your own.
Whether or not RTK actually does this has not been established. I would be glad to see some proper benchmarks done on the actual difference this tool makes (not some meaningless "up to 90%" type of language).
I ran a bunch of benchmarks and there are proven ways to reduce tokens while achieving the same results (finding the same CVEs / finding the same bugs in CRs, etc...).
See https://maki.sh, it's my own little proof.
anyway, what's the real pitch on why i should move on from opencode to maki?
I’m not OP, but parent comment and linked site https://maki.sh talk about token reduction.
In another comment you said "I can't get back to other agents". What gives? Feels like I completely misunderstood what Maki is.
As much as I hate to admit, T the tools you provide, the descriptions, and prompts, all amount to pretty big changes in experience, even using the same models.
Also -- can you make Maki enforce the underlying LLM to use stuff like fd/rg and not always default to find/grep, for example? And stop trying to do bash-isms in a zsh system?
Using fd/rg sounds interesting, honestly it would require little tweaks to the bash tool lua plugin, either add to the description to prefer these binaries instead or something like that.
In general though I much prefer "advising" and encouraging the LLM to use the native tools like grep l/glob, they are implemented to be super fast, and you will get better parser output.
Can you add an indicator to show whether the tool is currently running or not running (due to - no prompt, API error, waiting for permission etc)
But I like it, code processing is freaking fast.
Lord knows the DITA priesthood has been running low on rubes, so this new era is a godsend.
Re-coding all of your org's content into a verbose granular schema, that's what will fix these AI things. It's going to give your LLM superpowers! Semantic superpowers!
While everyone completely ignores the utter lack of coupling between the actual language and whatever nonsense is in the element / structure naming. Or the fact that every single thing has to go through some horrible 1990s era parser, which breaks constantly, and now everyone's shovelling the full markup into the very tiny confused mouth of the AI. Or that now everyone needs specialized software to display anything. Or the everything.
My dudes, the thing you're trying to do with this stuff is already done in the vectorizations. You can use math for a lot of it now, instead of someone hand coding "poplar" as "tree" in a totally flat tree structure.
They only float up if people create things like RTK and other people try them though.
It's fair to sit this one out and let others figure out if it's worth it or not but tools like RTK, Headroom, caveman mode and others do reduce input and output tokens that need to be processed, and for local LLMs that can have measurable speedups. Whether or not that ultimately hurts the resulting output I don't have enough data to say, but I am happy to play with them to find out.
If a tool like rtk improves token efficiency, but has some negative impact on quality, should Anthropic integrate it immediately? Where is the line? This kind of decision is arguably better left to the user.
What they should maybe do, is have a parameter similar to effort level, that allows the user to opt into native features for token minimizing. Make the tools available but leave the choice of the fidelity/savings tradeoff up to the user.
I've had a few times where removing all custom instructions that I started using with model N-2 made model N perform way better, so I'm very suspicious of everything that changes how the model works, it's easy to get degraded performance silently and suddenly you're paying latest Opus costs for 6 months old Sonnet performance.
[1]: https://github.com/JuliusBrussee/caveman
My criteria are: if it’s not in a harness it’s probably not that good (the best ideas float up to Codex/Claude imo) and any GitHub advertising some percent of token savings is not to be trusted.
It’s hard to avoid the snake oil and I hope people start thinking critically on this stuff.