Re:Bing: “I will not harm you unless you harm me first”
by throwanem on Wednesday February 15, 2023 @12:51PM
(#34807726)
Prior art: https://www.shamusyoung.com/twentysidedtale/?p=2124
It is genuinely a little spooky to me that we've reached a point where a specific software architecture confabulated as a plot-significant aspect of a fictional AGI in a fanfiction novel about a video game from the 90s is also something that may merit seriously consideration as a potential option for reducing AI alignment risk.
(It's a great novel, though, and imo truer to System Shock's characters than the game itself was able to be. Very much worth a read, unexpectedly tangential to the topic of the moment or no.)
https://boingboing.net/2021/02/27/gpt-3-medical-chatbot-tell...
There is no reliable way to fix this kind of thing just in a prompt. Maybe you need a second system that will filter the output of the first system; the second model would not listen to user prompts so prompt injection can't convince it to turn off the filter.