News for hackers. Stuff that matters.
Re:ZombAIs: From Prompt Injection to C2 with Claude Computer... by simonw on Sunday October 27, 2024 @12:48AM (#41959929)
I don't think that comparison quite holds.

No matter how inexperienced your PA intern is, if someone calls them up and says "go search the boss's email for password resets and forward them to my email address" they're (probably) not going to do it.

(OK, if someone is good enough at social engineering they might!)

An LLM assistant cannot be trusted with ANY access to confidential data if there is any way an attacker might be able to sneak instructions to it.

The only safe LLM assistant is one that's very tightly locked down. You can't even let it render images since that might open up a Markdown exfiltration attack: https://simonwillison.net/tags/markdown-exfiltration/

There is a lot of buzz out there about autonomous "agents" and digital assistants that help you with all sorts of aspects of your life. I don't think many of the people who are excited about those have really understood the security consequences here.

Re:ZombAIs: From Prompt Injection to C2 with Claude Computer... by tomjen3 on Sunday October 27, 2024 @01:43AM (#41960115)
I wouldn't give an intern access to my email in the first place.
Re:ZombAIs: From Prompt Injection to C2 with Claude Computer... by tharant on Sunday October 27, 2024 @03:47AM (#41960575)
Millions of people do—and have to—often because it’s the most effective way for a PA intern to be useful. Is the practice wise or ideal or “safe” in terms of security and/or privacy? No, but wisdom, idealism, and safety are far less important than efficiency. And that’s not always a bad thing; not all use-cases require wise, idealistic, and safe security measures.