Re:Is Opus 5.5 nerfed? Thinking patterns changed on November 3
by myworkaccount2 on Tuesday November 10, 2026 @03:49PM
(#50446130)
How do we know that they are not targeting this specific "benchmark" and serving it the same quality model?
> livenerf makes everything else deterministic: frozen prompts
One way to test my theory would be to run the benchmark yourself. (This catches if they are targeting this one account running the benchmark, have to say kind of unlikely...)
The other would be to see if something changes when you include this in your prompt: (this is the system prompt used by this test)
```You are completing a task that is graded automatically and exactly.
Work carefully. Follow the output format in the task precisely: graders do not read anything outside the requested format.```
To control for changes caused by the addition of this extra prompt itself, we could try to add some other piece of text that is semantically the same in meaning. (wont be perfect but maybe its close?)
To isolate impacts further we can do the same test with other non-nerfed models. (we ask, is Opus 5.5 the only model that is suddenly doing a lot better after i add this magic string in my system prompt, or will gpt5.6/quen 3.8 etc get better too)
> livenerf makes everything else deterministic: frozen prompts
One way to test my theory would be to run the benchmark yourself. (This catches if they are targeting this one account running the benchmark, have to say kind of unlikely...)
The other would be to see if something changes when you include this in your prompt: (this is the system prompt used by this test)
```You are completing a task that is graded automatically and exactly. Work carefully. Follow the output format in the task precisely: graders do not read anything outside the requested format.```
To control for changes caused by the addition of this extra prompt itself, we could try to add some other piece of text that is semantically the same in meaning. (wont be perfect but maybe its close?)
To isolate impacts further we can do the same test with other non-nerfed models. (we ask, is Opus 5.5 the only model that is suddenly doing a lot better after i add this magic string in my system prompt, or will gpt5.6/quen 3.8 etc get better too)