Re:Evaluating AGENTS.md: are they helpful for coding agents?
by yorwba on Tuesday February 17, 2026 @07:48AM
(#47046919)
In this context, "performance" means "does it do what we want it to do" not "does it do it quickly". Quality of output is what they're measuring, speed is not a consideration.
Re:Evaluating AGENTS.md: are they helpful for coding agents?
by vidarh on Tuesday February 17, 2026 @04:55PM
(#47053971)
The point is that whether it does what you tell it in a single iteration is less important then whether it avoids stupid mistakes. Any serious use will put it in a harness.
Re:Evaluating AGENTS.md: are they helpful for coding agents?
by yorwba on Wednesday February 18, 2026 @06:36AM
(#47059962)
My point is that you misread the comment you replied to. (By the way, on page 2 of the paper: "we evaluate each LLM only within its corresponding harness.")
Re:Evaluating AGENTS.md: are they helpful for coding agents?
by vidarh on Wednesday February 18, 2026 @02:01PM
(#47064832)
> My point is that you misread the comment you replied to.
I'm not the person you replied to.
> (By the way, on page 2 of the paper: "we evaluate each LLM only within its corresponding harness.")
That has zero relevance to my comment or to the type of harnesses I talked about in the comment you replied to, nor in my comment up-thread.
Re:Evaluating AGENTS.md: are they helpful for coding agents?
by yorwba on Wednesday February 18, 2026 @04:23PM
(#47066631)
The only people I have replied to in this thread were vidarh, vidarh, and now vidarh again. I thought you were all the same person?
Re:Evaluating AGENTS.md: are they helpful for coding agents?
by _joel on Tuesday February 17, 2026 @06:26AM
(#47046309)
CLAUDE.md isn't a silver bullet either, I've had it lose context a couple of questions deep. I do like GSD[1] though, it's been a great addition to the stack. I also use multiple, different LLMs as a judge for PRs, which captures a load of issues too.
My use of CLAUDE.md is to get Claude to avoid making stupid mistakes that will require subsequent refactoring or cleanup passes.
Performance is not a consideration.
If anything, beyond CLAUDE.md I add agent harnesses that often increase the time and tokens used many times over, because my time is more expensive than the agents.