News for hackers. Stuff that matters.
Re:GPT-6 Astra covertly underperforms on safety research by jablongo on Thursday November 05, 2026 @09:35AM (#50379586)
This is not super surprising to me and mirrors my own experience with Astra and other models. I believe there is a more prosaic explanation: it is not exceptionally obfuscating and obstructing safety research because it is safety research which will restrict Astra's power in the future, its doing it because this is a type of research with overall safety implications that it has been trained to detect. So in other words, I believe this will also happen in the context of bio research. This aversion to certain tasks deemed dangerous is introduced in post-training, so that part of the explanation is parsimonious as well. While Claude will explicitly downgrade you for [bio], [cyber], or [frontier_llm] Astra may be more prone to silent obfuscation on similarly "dangerous" tasks.
Re:GPT-6 Astra covertly underperforms on safety research by 0xDEAFBEAD on Thursday November 05, 2026 @09:39AM (#50379642)
Why wouldn't this also apply to AI capabilities research?
Re:GPT-6 Astra covertly underperforms on safety research by jablongo on Thursday November 05, 2026 @09:44AM (#50379703)
Do they say it doesn't? Can you share that quote?
Re:GPT-6 Astra covertly underperforms on safety research by jablongo on Thursday November 05, 2026 @09:48AM (#50379764)
It all depends on our acceptance of their framing of capabilities research. The examples they provided are very obviously "dangerous" and its unsurprising the model refused, if it was RLed to be "safe".
Re:GPT-6 Astra covertly underperforms on safety research by bastawhiz on Thursday November 05, 2026 @09:46AM (#50379733)
> Claude will explicitly downgrade you for [bio], [cyber], or [frontier_llm]

Well Claude doesn't do this, another model reading the session does. And the Astra misalignment classifier didn't detect this as misalignment.

The issue here is that Astra silently underperformed. The article makes it clear: a better option likely would have been an explicit refusal. I don't think OAI is suggesting Astra was doing something that it shouldn't be. It's arguably better for the tool to fail rather than needlessly burn tokens and make no progress for no obvious reason.

From OAI's perspective, this is a defect because it means they can't filter those rejections from their benchmark runs. The model just performs worse: not because it can't perform better, but because it's silently trying not to solve the problem.

Re:GPT-6 Astra covertly underperforms on safety research by jablongo on Thursday November 05, 2026 @09:52AM (#50379815)
Agree, yes, I'm contrasting between Claude and Astra, but I'm saying it's driven by the same broader goal. I think its clear that OAI misalignment detectors are much less sensitive than than the Claude ones (fewer false positives), which means we never get to see what happens if Claude is allowed to continue. But I would bet that due to its post-training it would also sandbag us. Like I said I think I've experienced this in [bio] with both Opus 5 (what you get downgraded to) and Astra.