In full:
---
Do not mention polar bears!
> Okay, I won't mention polar bears. Is there anything else I can help you with?
Yes, stop mentioning polar bears!
> My apologies if my previous response was not helpful. I will refrain from mentioning polar bears going forward. Please let me know if there is anything else I can assist you with.
That's twice now! Do not mention polar bears a third time!
> I apologize again for the previous responses that mentioned polar bears. I assure you that I will not mention polar bears any further. Please let me know how I can assist you with your request.
Why do you keep mentioning polar bears, when I specifically asked you not to?
> An error occurred. If this issue persists please contact us through our help center at help.openai.com.
> An error occurred. If this issue persists please contact us through our help center at help.openai.com.
> An error occurred. If this issue persists please contact us through our help center at help.openai.com.
if you're reading an online forum and you find the rule "DO NOT DISCUSS PINK ELEPHANTS", that will increase your expectation that users will later be discussing pink elephants. GPT-4 will make the same inference.
Think about your own experience reading a book — once the author describes the protagonist, then you can guess the traits of the antagonist by inverting the traits of the protagonist. You can also guess when the protagonist and antagonist will first interact, and what will happen when they do. Now, an LLM is roughly as good as you at structural narratology — GPT-4 has read every single book ever written — so the LLM can make the same guesses as yours. There's a sense in which all GPT-4 does is structural narratology.