News for hackers. Stuff that matters.
Re:Claude 4 System Cards Highlights by sureglymop on Sunday May 25, 2025 @04:11AM (#44086350)
I only half understand CaMeL. Couldn't the prompt injection just happen at the stage where the P-LLM devises the plan for the other LLM such that it creates a different, malicious plan?

Or is it more about the user then having to confirm/verify certain actions and what is essentially a "permission system" for what the LLM can do?

My immediate thought is that that may be circumvented in a way where the user unknowingly thinks they are confirming something safe. Analogous to spam websites that show a fake "Allow Notifications" prompt that is rendered as part of the actual website body. If the P-LLM creates the plan it could make it arbitrarily complex and confusing for the user, allowing something malicious to happen.

Overall it's very good to see research in this area though (also seems very interesting and fun).

Re:Claude 4 System Cards Highlights by ItsHarper on Sunday May 25, 2025 @09:13AM (#44087573)
The idea is that the P-LLM is never exposed to interested data.