Re:Anthropic scientists hacked Claude's brain – and it noticed
by mike_hearn on Friday October 31, 2025 @06:41AM
(#45770516)
No, the thinking trace is generated tokens but demarcated by control tokens to suppress them from API output. To inject things into that you'd just add words, which is what their prefill experiment did. That experiment is where they distinguish between just tampering with the context window to inject thoughts vs injecting activations.
[ Parent
| on HN ]
Re:Anthropic scientists hacked Claude's brain – and it noticed
by andy99 on Friday October 31, 2025 @08:32AM
(#45771314)
What I was wondering is, do the injections cause the thinking trace to change (not whether they actually typed text into the thinking trace) and then the model “reflects” on the fact that it’s thinking trace has some weird stuff in it, or do these reflections occur absent any prior mention of the injected thought.
Re:Anthropic scientists hacked Claude's brain – and it noticed
by mike_hearn on Friday October 31, 2025 @09:38AM
(#45771856)
Well, the paper makes no mention of any separate hidden traces. These seem to be just direct answers without any hidden thinking tokens. But as the thinking part is just a regular part of the generated answer I'm not sure it makes much difference either way.