News for hackers. Stuff that matters.

A comment on Is Opus 5.5 nerfed? Thinking patterns changed on November 3. Parent comment

Re:Is Opus 5.5 nerfed? Thinking patterns changed on November 3 by nsingh2 on Monday November 09, 2026 @10:32AM (#50427006)
It's hard to tell, but I think they also pin the harness version? Updates to Claude Code can also have big effects on perf, and Claude Code updates a lot.
[ Parent | on HN ]
Re:Is Opus 5.5 nerfed? Thinking patterns changed on November 3 by arecurrence on Monday November 09, 2026 @01:15PM (#50429282)
This is a good point. Importantly, Claude Code updates change the system prompt and that has both direct and very broad effects on the output. Livenerf hasn't changed the Claude CLI version in 2 months.

They also have this comment in getting started: "Pin the CLI. This is not optional: a Claude Code update changes the harness, and a changed harness looks exactly like a changed model. Turn off auto-updates and write down the version you're pinning"

[ Parent | on HN ]
Re:Is Opus 5.5 nerfed? Thinking patterns changed on November 3 by pythonaut_16 on Monday November 09, 2026 @03:14PM (#50431298)
Which seems both valid and wild. Valid that if you want to compare model performance you need to limit the variables.

Wild because as an end user I don't care what Anthropic changed to affect performance, I only care that Anthropic caused degraded performance.

[ Parent | on HN ]
Re:Is Opus 5.5 nerfed? Thinking patterns changed on November 3 by Rapzid on Monday November 09, 2026 @04:01PM (#50432073)
It doesn't even matter if you pin them. Claude Code and Codex have "experiments" baked in and they can change your cohort remotely.

So your harness behavior can change without updating it.

Fun.

[ Parent | on HN ]