Re:The Waluigi Effect: an explanation of bizarre semiotic ef...
by astrange on Monday March 06, 2023 @06:51PM
(#35049571)
The problem there may be that you've constructed a latent space which has a shape of the thing to avoid. That's the problem with prompts like "don't think of a pink elephant" - it has to know what those words mean. Better to not label it so there isn't a way in there, if you can.
And of course humans also have this issue.
The most compelling point the author makes is that once the AI learns a shape (e.g. the shape of Luigi in personality space), it’s just a bit flip to invert that shape. So all an attacker needs to do is flip that one bit.