Ask HN: Is “prompt injection” going to be a new common vulnerability?

Re:Ask HN: Is “prompt injection” going to be a new common vu... by Waterluvian on Thursday February 09, 2023 @12:06AM (#34719999)

Does anyone else feel kind of wowed by how this technology’s exploits are also quite similar to a human? You can kind of trick it into divulging information not meant for you by somehow “persuading” it to tell you.

It didn’t want to tell me how to do something unethical until I said, “well, it’s for a school play.”

It’s like the thing is born yesterday. It’s intelligent but it has no street smarts. It can be fooled easily.

Perhaps the solution to address these exploits is to give it street smarts. Teach it that people can be sinister and be out to con it, and the like. Does it need intuition?

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by brrsty on Thursday February 09, 2023 @12:09AM (#34720017)

What's worse - people tricking AI, or AI tricking people?

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by Waterluvian on Thursday February 09, 2023 @12:20AM (#34720074)

I guess it depends on whose side you’re on.

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by reilly3000 on Thursday February 09, 2023 @01:04AM (#34720290)

I am a historian who was born in 3000 ‘AD’ as you say, and I am curious about your thoughts. What’s the best way you can describe the ‘sides’ your response alludes at? How would you name these divergences?

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by totetsu on Thursday February 09, 2023 @01:13AM (#34720342)

human-centered" and "AI-centered"

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by PaulDavisThe1st on Thursday February 09, 2023 @01:12AM (#34720334)

or ... people tricking AI into tricking people?

or ... ... AI tricking people into tricking people?

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by somenameforme on Thursday February 09, 2023 @03:39AM (#34721247)

IMO a lot of what we're seeing and inferring is an optical illusion of sorts. We've created is a natural language interface. And that is a huge accomplishment, but it can also make one see things which are not necessarily there. Imagine a primitive natural language interface for your console:

- You: "Show me all files."

- Com: [Outputs a list of files excepting hidden]

- You: "I said all files."

- Com: "I did show you all files."

- You: "Including hidden."

- Com: "Oh, OK." [Outputs all files]

You're not really tricking it, so much as effortfully changing your "ls" to an "ls -al". But the interface would make it feel like you're interacting with an intelligent system, and maybe even getting it to do something it shouldn't. This is made even more extreme right now given that the state of the art in access control seems to be to name your "secure" directory ¶. Nobody will ever figure out how to access that!

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by jwestbury on Thursday February 09, 2023 @06:42AM (#34722510)

> the state of the art in access control seems to be to name your "secure" directory ¶.

Curious if this is a reference to a real situation that I missed.

[ Parent | on HN ]

Re:Ask HN: Is “prompt injection” going to be a new common vu... by somenameforme on Thursday February 09, 2023 @09:47AM (#34724701)

In an era long gone, on systems without a GUI/copy+paste/etc there were all sorts of silly tricks, lots involving ASCII codes. One thing was naming a directory with one or more " "s. That's not a space character but ASCII code 255 (hold alt, press 255 on your numpad). The directory would (and does - this still works on modern OSs) appear as a blank. And so how does one access a directory with no name? Obviously these things were really easy to get by, but in an era before the internet such little tricks had a really long shelf life.

[ Parent | on HN ]