News for hackers. Stuff that matters.

A comment on AI-powered Bing Chat spills its secrets via prompt injection attack. Parent comment

Re:AI-powered Bing Chat spills its secrets via prompt inject... by Tv9m on Tuesday February 14, 2023 @12:06AM (#34785543)
> Is prompt injection even a problem worth worrying about?

It depends what API access the AI has. If it's just a chat bot, prompt injection can only reveal facts about its language model. But if the AI has POST access to something, depending on what it is, prompt injection can set off arbitrary human-caused disasters.

[ Parent | on HN ]
Re:AI-powered Bing Chat spills its secrets via prompt inject... by nradov on Tuesday February 14, 2023 @12:37PM (#34792895)
That's not the correct way to do security vulnerability analysis. If an API call can cause a disaster then fix the API. Whether the API consumer is an AI or some other type of system is irrelevant.
[ Parent | on HN ]
Re:AI-powered Bing Chat spills its secrets via prompt inject... by Tv9m on Sunday February 26, 2023 @02:12PM (#34948386)
> If an API call can cause a disaster then fix the API

By "API" I'm not referring just to publicly facing REST endpoints. I mean things like shell access for system maintenance, that normally only human professionals like you would be given. In the future it's not clear that humans will be able to dominate that role forever.

Hopefully the issues will be recognized while LLM-based agents are still only serving as retrieval systems.

[ Parent | on HN ]