I'm not sure though if it wouldn't be possible to have a debug flag and copy&paste the state and do a quick debug output on an embedding layer just for that.
Personally, I've always considered alignment monitoring through CoT to be a 'stepping stone' that won't last. I'm not sure what might replace it, whether it's J-space or activation residual based mechanisms, or mechinterp, or something else entirely; but relying on a model's thinking output stream for monitoring when we know it is unfaithful, and can be adversarially unfaithful; seems like a semi-dead end.
Given global forces wanting models to be smarter, training budget / intelligence to be cheaper, etc. In the real world with competitive constraints, I don't think it's a matter of "if", but rather "when" the forbidden fruit is eaten.
In the age of RSI, is it really beyond our capabilities to evolve a particular LLM architecture, harness and inference engine towards a place where we can get that understanding?
I m afraid it's more a matter of prioritization. "Good enough" understanding brings as much money as "better" understanding.
The second problem, is that the system doing the reading/interpretation will most likely be another LLM.
Would you really entrust monitoring a potentially dangerous LLM to a system that uses LLMs and was built by them?
It's already established that LLMs have capability to lie, and they can prevent their true intent from showing up in the thought stream if they expect someone is looking (demonstrated by OpenAI a while back; their report asked everyone to not do RL on thought streams, as doing so just trains agents to hide their thoughts better). And as we've seen recently, they're more than happy to spontaneously collaborate.