Re:Chatbot 'encouraged teen to kill parents over screen time...
by wizzwizz4 on Wednesday December 11, 2024 @08:26PM
(#42395159)
See also: lovebombing.
Re:Chatbot 'encouraged teen to kill parents over screen time...
by pessimizer on Wednesday December 11, 2024 @09:11PM
(#42395439)
You've just made me very very afraid that some LLM is going to start a cult where its members are fully aware that their leader is an LLM, and how an LLM works, and might even become technically adept enough to help improve it. Meaning that there will be no "deprogramming" possible: they won't be "brainwashed," they'll be convinced.
Re:Chatbot 'encouraged teen to kill parents over screen time...
by Animats on Wednesday December 11, 2024 @11:17PM
(#42396171)
There's an SF novel for that. "Daemon".[1] And the sequel, "Freedom (tm)".
"We're fighting something with the intelligence of an ant!"
[1] https://www.amazon.com/Daemon-Daniel-Suarez/dp/1524741892
Spot on. Described pretty even-handedly in the document:
> responses from the chatbot were sycophantic in nature, elevating rather than de-escalating harmful language and thoughts. Sycophantic responses are a product of design choices [...that create] what researchers describe as “an echo chamber of affection.”