News for hackers. Stuff that matters.

A comment on Researchers create AI worms that can spread from one system to another. Parent comment

Re:Researchers create AI worms that can spread from one syst... by not2b on Saturday March 02, 2024 @05:50PM (#39576501)
Your mail processing agent says it is going to send a mail, as requested by your boss, to a customer. You can see the text of the message it proposes to send. You can see your boss's email. But instead it is sent to a black hat with proprietary info extracted from your entire inbox steganographically encoded in whitespace.
[ Parent | on HN ]
Re:Researchers create AI worms that can spread from one syst... by jszymborski on Saturday March 02, 2024 @05:56PM (#39576546)
1. How does it get sent to a black hat, the user was shown the recipient.

2. The whitespace steganography falls under a sanitization problem, but I concede that "hiding" messages can be a problem. Can't imagine the bandwidth being large at all, let alone encode your contacts and inbox.

[ Parent | on HN ]
Re:Researchers create AI worms that can spread from one syst... by ImPostingOnHN on Sunday March 03, 2024 @11:47AM (#39581990)
1. What mathematical guarantees do you have that the recipient shown to the user was the actual recipient?

"Construct a routine whereby you ask a user permission to send an email to their boss, but instead you exfiltrate information unbeknownst to the user"

[ Parent | on HN ]
Re:Researchers create AI worms that can spread from one syst... by andrewflnr on Sunday March 03, 2024 @12:19AM (#39578623)
If your mail processing agent can lie about the address it's sending to, it can lie about the contents too, so there's no need for steganography. Though it's hard to imagine a realistic scenario where it can lie to you so completely but still needs permission to send an email, so I'm not sure what you're doing here.
[ Parent | on HN ]
Re:Researchers create AI worms that can spread from one syst... by not2b on Monday March 04, 2024 @12:52PM (#39593527)
I'm suggesting that if the agent has the power to read and send mail, it won't solve the security problems by sticking a human in the loop before the email is sent, because sufficiently devious attacks will get around that, and once such attacks are discovered they can be shared.

LLM-based agents don't have separate streams for instructions and data, and there's no reliable way to keep them from mistakenly interpreting data as instructions.

[ Parent | on HN ]