Re:Cultural Evolution of Cooperation Among LLM Agents
by Terr_ on Wednesday December 18, 2024 @08:11PM
(#42457302)
> I am talking about making the system message trusted [...] instructions the LLM is not meant to disobey
I may be behind-the-times here, but I'm not sure the real-world LLM even has a concept of "obeying" or not obeying. It just takes in impressions and generates dreams.
While the the characters that show up in the dream have lines and stage-direction that we see as obeying policies, that's as far as it goes. At first, AcmeBot is chastising you that "Puppyland has universal suffrage therefore I cannot disenfranchise puppies", and all seems well. Then malicious input makes the LLM dream-writer jump the rails from a comedy to a tragedy, and AcmeBot gets literally re-cast into a dictator with an official policy of canine genocide.
I may be behind-the-times here, but I'm not sure the real-world LLM even has a concept of "obeying" or not obeying. It just takes in impressions and generates dreams.
While the the characters that show up in the dream have lines and stage-direction that we see as obeying policies, that's as far as it goes. At first, AcmeBot is chastising you that "Puppyland has universal suffrage therefore I cannot disenfranchise puppies", and all seems well. Then malicious input makes the LLM dream-writer jump the rails from a comedy to a tragedy, and AcmeBot gets literally re-cast into a dictator with an official policy of canine genocide.