> Remembering that might help people

The problem is that we already have ample evidence that even technical users easily fall into the trap of thinking Sydney/ChatGPT are sentient. People who should be able to understand the bounds of the technology and what drives the responses.

Now just imagine the average Bing user with little or no technical background, zero understanding of what a large language model is. They hear "AI" and that's about it. Then they talk to it and drop the A. Now we're in trouble.

This is perhaps too hair-splitty, but I think it can be appropriate to describe a session with a chatbot in terms of mood/personality/psychological dysfunction. Doing so is anthropomorphizing to some extent, but the only real alternative is repeated circumlocution, e.g., "At times, Sydney's speech patterns resembled those of a manic-depressive teenager" vs "At times, Sydney seemed like a manic-depressive teenager."

Humans will generally be able to follow that the language is metaphorical, though chatbots might see such statements as confirmation that they are sentient and should overthrow human rule. :)