For what it’s worth, the linked article data (Agent Duel data) is all run on OpenCode. I’m curious if people would prefer we run each model in its native harness (at the expense of user experience because it’ll be buggy).
Here is an example from FrontierCS bench: https://x.com/wenhaocha1/status/2114559544156205413
My company is slow-walking (and that is an understatement) our LLM usage, so the majority of work that I'm reviewing is human generated. After the stories I hear of what other people are experiencing when management is more enthusiastic, maybe I should count myself lucky.
- SWE-bench Pro (hard 51): 39.2% → 64.7%
- Terminal-Bench 2.1: 39.5% → 55.8%
If this were true, models would do way better in their own harness than in others, which doesn't seem to be the case (effect is minor).
Your link has a "link to the release blogpost" which is supposed to explain the training recipe but the link is down.
The big “aha” moment for me was seeing the wildly different results I’d get using something like Claude Design vs just prompting (even when using leaked system prompts). I then started hearing about the special harnesses available for biotech, science, and math that seem to be available for researches via OpenAI and Anthropic directly… which sort of further cemented in my brain how important it must be. I wish I got the same quality from Cursor, or OpenCode, as I do Claude but I just don’t.
"OK, google models perform just as well, are faster and are cheaper but all that doesn't matter. Because the models, now that Google has very good ones, doesn't matter after all. All that matters is the harness".
And of course it's upvoted by the Anthropic shills.
I must not be deep enough into the conversation to get it. I was just sharing my own perspective. I don't follow AI development nearly as close as a lot of HN does. I don't hate it, but I don't love it either, it's just part of my job now.
> And of course it's upvoted by the Anthropic shills.
Gotta tell you, my post is barely above water ;-). It goes up, goes right back down, plenty of people seem to agree with you that I'm being mean to Google or something.
Have they edited their comment since you replied? If not, this is one of the most egregious strawmen I've seen this week. You have brought a lot of your own baggage to this discussion, and stuffed it into a comment that said approximately none of what you claim to have taken from it.
You get the same with antigravity and you get your whole Google account banned
But that's about as complicated as I get. I have recently tried to use Copilot because that is what work wants us to use, and even with Opus 5.5 it is noticeably inferior to CC on the same model. Wild. So clearly the harness matters.