wek

I agree with this. Why get locked in at the harness layer. I want my workspace to be independent of my coding agents and models so I can use multiple and switch as I wish

This is cool! We have a similar approach in Nimbalyst... agent makes extensions that are human editable and agent editable ... includes Jupyter, markdown, csv, excalidraw, code, pdfs, browser. Its so nice not to have to app switch all the time. MIT as well. Maybe we should join forces?

Thank you for this. This seems like a promising approach to agent permissioning. What are the performance implications?

From their abstract: "Our findings reveal that despite continuous development activity and growing codebase complexity of the agent harnesses, there is no statistically significant improvement in SWE-bench benchmark score (i.e., resolve rates of bugs) across releases for a given fixed LLM version. Worse, later agent harness versions consume nearly double the computational tokens and tool calls without corresponding quality gains. We explain this paradox from two angles: at the project level, we identify the development patterns (e.g., feature additions, fix-heavy releases, scattered small changes) correlating with quality fluctuations, while at the architecture level, we localize regressions to specific high-risk architectural components. Our findings call for a new practice of quality assurance in the development of agent harnesses."

Exactly. Expertise is frequently required to steer, correct, rework, know what is good and what isn't. This happens 500 times a day on both the micro level and the architecture level.

Use Nimbalyst (I work on it). Multi-sessions, visual editing, open-source

I have Max subscription to both Claude Code and Codex. I use both fulltime throughout the day. I have them check each other's work, and route tasks they are best at to the right one.

The checking each other's work is so helpful.

New features, mockups, I use Claude Code. Deep fact-checking, I use Codex.

My harness matters at least as much as the model. I use the best harness and then invest a ton in it

The article is correct to emphasize the importance of definition and that this can be a bottleneck. But it is incorrect to show the layered documentation and development lines taking just as long as they did pre-coding-agents.

Our team is coding much much faster at high quality then we were before coding agents. As this commenter says, if you empower individuals and small teams to define, document and they set up good practices and harnesses, they can accelerate dramatically.

Well said. It is so much better for when the human can see what the LLM has changed and iterate on it with the LLM, making their own changes. Markdown is better for that.

I kind of felt the same way reading the article! It felt so unusual to encounter someone who is both smart and humble and willing to admit they were learning. And I was happy to encounter it and sad that I was so surprised by it.

Congratulations! I think that a lot of the value will be in the judgement of the maintainer about the marginal next feature (and saying no to all those other features) ... if software is a stream then the value is in what gets into the stream. That is an AI resistant value and if you can provide it for your project.

What an excellent article by a smart, humble, still-learning person!

Favorite quote:" There are a whole bunch of reasons I’m not scared that my career as a software engineer is over now that computers can write their own code, partly because these things are amplifiers of existing experience. If you know what you’re doing, you can run so much faster with them. [...]

I’m constantly reminded as I work with these tools how hard the thing that we do is. Producing software is a ferociously difficult thing to do. And you could give me all of the AI tools in the world and what we’re trying to achieve here is still really difficult. [...]"

And as of today Nimbalyst is open source!

This is on our wish list for a custom extension. Many users have asked for it. If someone in the community wants to take a crack at it....

Thanks! A good way to learn the platform is by building an extension.