Also, once you give it this general access, it opens up essentially infinite directions for the model to go to. Repeatability and testing become very difficult in that situation. One time it may write a bash script to solve the problem. The next, it may want to use python, pip install a few libraries to solve that same problem. Yes, both are valid, but if you desire a particular flow, you need to create a prompt for it that you'll hope it'll comply with. It's about shifting certain decisions away from the model so that it can have more room for the stuff you need it to do while ensuring that performance is somewhat consistent.
For now, managing the context window still matters, even if you don't care about efficient token usage. So burning 5-10% on re-writing the same API calls makes the model dumber.
The token are not wasted, because I rewind to before it started building the tool. That it can build and manipulate its own tools to me is the benefit, not the downside. The internal work to manipulate the tools does not waste any context because it's a side adventure that does not affect my context.
If you mean manually, that's still interesting, but that kind of feels like the same thing to me. The idea is - don't let the agent burn context writing tools, it should just use them. Isn't that exactly what yours is doing? Instead of rewinding to a snapshot, I have a separate code base for it. As tools get more complex, it seems nice to have them well-tested with standardized input and output. Generating tools on the fly, rewinding, and using tools is just the same thing. You even would need to provide some context that says what the tool is and how to use it, which is basically what the mcp server is doing.
I'm basically saying this except I currently don't give the agent a tool yet to do it automatically because it's not really RL'ed to that extend. So I use the branching and compaction functionality of my harness manually when it should do that.
> If you mean manually, that's still interesting, but that kind of feels like the same thing to me.
It's similar, but it retains the context and feels very naturally. There are many ways to skin the cat :)
Claude Code's --permission-prompt-tool flag is a good example. You point it at an MCP server, and every permission request goes through that server instead of a local prompt. The server can do whatever: post to Slack, require 2FA, log to an audit trail. Instead of "allow all DB writes" or "deny all," the agent requests approval for each mutation with context about what it's trying to do.
MCP is overkill for "read a file" but valuable when you need the agent to ask permission, report progress, or hand off to another system mid-task.
My agent writes its own glue code so the benefit does not seem to really exist in practice. Definitely not for coding agents and increasingly less for non coding agents too. Give it a file system and bash in a sandbox and you have a capable system. Give it some skills and it will write itself whatever is neeeded to connect to an API.
Every time I think I have a use case for MCP I discover that when I ask the agent to just write its own skill it works better, particularly because the agent can fix it up itself.