certainforest

+1

Hey, Jasmine here (one of the authors) -- that's an interesting idea!

There's an interesting exploration of this here: https://www.lesswrong.com/posts/HEzNZ9gvgYwT3aZFS/role-embed....

Curious if you have additional thoughts, and thanks for reading!

Yes, this is something we're thinking about!

Thanks for reading.

Hey, Jasmine here -- it's a good point, I'm generally more concerned by agentic jailbreaks (e.g. unauthorized purchases, leaking sensitive data) than GPT making inappropriate comments.

In our case, we found that simply acting like a user is enough to trick LLMs into sharing passwords, private files, etc.

(On a related note, here's one where they hack a smart home with email invitations: https://sites.google.com/view/invitation-is-all-you-need/hom...)