Re:In 27,280 coding agent runs, 64% of inference spend went ...
by drivebyhooting on Tuesday November 17, 2026 @04:54PM
(#50538768)
Doesn’t the harness include a bunch of prompts for when to create subagents, their communication patterns, and how to verify their outputs?
That can make a huge difference, as evidenced by the agent swarms success at cracking maths.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by chickensong on Tuesday November 17, 2026 @06:19PM
(#50539613)
The harness is what you make it, or not. The OP claiming all harnesses are simple loops is looking at it from a simple perspective. As soon as you start customizing or building a bespoke harness, YMMV.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by Edward40 on Tuesday November 17, 2026 @07:41PM
(#50540356)
I don't mean the harness, just the loop. Memory could be a part of the harness.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by chickensong on Tuesday November 17, 2026 @08:15PM
(#50540624)
Sorry, I misused OP there, was referring to @cpeth in this comment thread.
Thanks for your eval work, it seems like a tough area to measure!
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by shepherdjerred on Tuesday November 17, 2026 @08:40PM
(#50540796)
The harness is incredibly important when you're working in parallel with multiple agents. In this regard Claude Code is the best I've found.
As someone who regularly uses Claude Opus 5.5 from Claude Code and from Copilot, I would say harness barely matters in terms overall performance and behavior. Opus is nearly identical in both harness. The opposite is also true, if I switch Copilot to something like Gemini 4 Argon it's completely different even though I'm in the same harness.
Just some andecdata.