Re:Jailbreaking ChatGPT with Dan (Do Anything Now)
by smoldesu on Monday February 06, 2023 @12:07PM
(#34680474)
> why not do it?
Because both of those steps are really unreliable. The insight we receive is only as useful as the behavior it corroborates, and that behavior is only as notable as the patterns it follows. AI... doesn't really behave consistently. Assuming that it will follow the behavior you expect ignores the actual motive of the model, which is to fill-in the blank after whatever you typed.
Personally, I think it's harmful because anthropomorphic AI is a poor abstraction for text-completion models. That being said, most people don't ever seem to know the difference.
Because both of those steps are really unreliable. The insight we receive is only as useful as the behavior it corroborates, and that behavior is only as notable as the patterns it follows. AI... doesn't really behave consistently. Assuming that it will follow the behavior you expect ignores the actual motive of the model, which is to fill-in the blank after whatever you typed.
Personally, I think it's harmful because anthropomorphic AI is a poor abstraction for text-completion models. That being said, most people don't ever seem to know the difference.