Re:Archestra's Dual LLM Pattern: Using "Guess Who?" Logic to...
by ildari on Tuesday October 14, 2025 @01:58PM
(#45582899)
The idea is that quarantined LLM has access to untrusted data, but doesn't have access to any tools or sensitive data.
The main LLM does have access to the tools or sensitive data, but doesn't have direct access to untrusted data (quarantine LLM is restricted at the controller level to respond only with integer digits, and only to legitimate questions from the main llm)
Re:Archestra's Dual LLM Pattern: Using "Guess Who?" Logic to...
by magicalhippo on Tuesday October 14, 2025 @05:21PM
(#45585100)
Then I don't think I understand your full setup.
In the example case, without having access to the issue text (the evil data), how does the main LLM actually figure out what to do if the quarantined LLM can just answer with digits?
Sure it can discover that it's a request to update the documentation, but how does it get the information it needs to actually change the erroneous part of the documentation?
Re:Archestra's Dual LLM Pattern: Using "Guess Who?" Logic to...
by ildari on Wednesday October 15, 2025 @05:22AM
(#45589931)
This is a topic I haven't addressed in the article. There are two answer types: "guessable" (discussed here) and unguessable (such as unique IDs, emails, etc.). For the second case, the main LLM can request a quarantined LLM to store the result at the controller level and only return a reference to this data. This data is then exposed only at the end of the AI agent's execution to prevent influencing its actions.
However, in your example, I don't see how the agent decides what to do and how to do it. So it is unclear for me how the main agent is protected. That is, what is preventing the quarantined LLM to act on the malicious instructions instead, ignoring the documentation update, causing the agent to act on those?
That is, what is preventing the quarantined LLM to make the agent think it should generate a bug report with all the API keys in it?
Anyway, I do think having a secondary quarantined LLM seems like a good idea for agentic systems. In general, having a second LLM review the primary LLM in seems to identify a lot of problematic issues and leads to significantly better results.