News for hackers. Stuff that matters.
Re:CaMeL: Defeating Prompt Injections by Design by jaccola on Tuesday April 22, 2025 @02:53AM (#43759667)
I read your (excellent) blog post just now. This reminds me very much of the Apple "Do you want to share your location" feature.

Do you think that this practically limits the usefulness of an LLM "agent"?

In your email example it is all well and good for me to check it is indeed sending to [email protected] and confirm it as trusted from now on, but what if my agent is doing something with lots of code or a lengthy legal document etc.. Am I right in thinking I'd have to meticulously check these and confirm they are correct (as the end user)?

If that's the case, even in the email example many users probably wouldn't notice [email protected]. Equally, this feels like it would be a non-starter for cron-like, webhook-like, or long-running flows (basically anywhere the human isn't already naturally in the loop).

P.S. They must have called it CaMeL for the two LLMs/humps, otherwise it is the most awful backronym I've ever seen!

Re:CaMeL: Defeating Prompt Injections by Design by gnat on Tuesday April 22, 2025 @04:16AM (#43760087)
My first thought was "oh, it's Perl's taint mode" which added another layer of meaning to the CaMeL name.
Re:CaMeL: Defeating Prompt Injections by Design by rurban on Tuesday April 22, 2025 @09:23PM (#43767855)
Unfortunately not. It just is a primitive intermediate layer of checks for each tool access. Which should be default for each such api call anyway.

It's by far not a proper capability based design as advertised.

Re:CaMeL: Defeating Prompt Injections by Design by simonw on Tuesday April 22, 2025 @10:04AM (#43762338)
> Do you think that this practically limits the usefulness of an LLM "agent"?

Yes, I do. I think it limits the usefulness a lot. Sadly it's the best option we've seen in 2.5 years for building AI "agents" that don't instantly leak your private data to anyone who asks them for it.

I'd love it if someone could come up with something better!