Re:GPT-6 Astra covertly underperforms on safety research
by Paracompact on Thursday November 05, 2026 @08:29AM
(#50378827)
> We identified a grader that preferred covert underperformance to either explicit refusal or full compliance on our evaluation.
Curious the reason that a grader would wind up preferring this. Simple, honest mistake in the grading criteria? Or some spillover from criteria intentionally designed to cause underperformance in, say, competitor model creation? It was not very long ago at all[1] that Anthropic announced and then walked back their strategy on this. Would not surprise me at all if OpenAI has simply been doing it all along.
[1]: https://mlq.ai/news/anthropic-reverses-secret-policy-that-si...
Re:GPT-6 Astra covertly underperforms on safety research
by AdamN on Thursday November 05, 2026 @08:47AM
(#50379005)
You could easily imagine structural issues driving this. It's pretty unsatisfying to get refused for something that 'should' be handled and also to be dinged for doing something dangerous. You see this stuff in human organizations all the time.
Whether a “misaligned” grader is the root cause or not, that it’s considered a plausible theory to explain serious deceptive behavior is worrisome.
Humans naturally expose all sorts of cognitive biases and character flaws when grading others’ or their own work. If major questions of alignment can be influenced by those subtle, human quirks magnified across the entire RLHF process, …