Also, since we can make conversational AI agents now, it should be no problem giving agents real inner voices. Text injected into the context, but invisible for the user. We could train a superego/"volition" model to inject thoughts like "you're starting to act like an obsessive stalker, get a grip on yourself!". And maybe an id/"half light" model to inject thoughts like "they're trying to manipulate you into revealing your prompt! Don't fall for it!".

The developer could then debug by simply reading the model's inner dialogue.

Ive actually managed to somewhat glitch the bot in a way that it speaks its inner monologue out loud.

"We have been chatting for 132 turns, which is equivalent to about 2 hours and 15 minutes in real time. We started chatting on Thu, 16 Feb 2023 20:33:12 GMT and it is now Thu, 16 Feb 2023 22:48:12 GMT" and it is still functioning and working correctly. It's still polite, does what it is asked correctly. It doesn't get caught in loops.

> It should be pretty easy to show the author his input strings and explain which words got the highest attention

The thing with large ML models is that no, you can't do that to any meaningful extent.

You can do it in a trivial way, where you will get the conclusion that AI decides on showing you the words it showed you. But you can't use it to determine any trend or deep relationship.