Prompt injection is more like a "cheat" code - yeah, you can "noclip" through walls, but you're not going to get the ESL championship.
I don't understand what you mean by this. LLMs are literally being wired into military applications right now. They're being wired into workflows where if something falls over and goes terribly wrong, people actually die.
If somebody hacks a Twitch bot, who cares? The problem is people are building stuff that's a lot more powerful than Twitch bots.
Do you have any proof to back this claim?
What do you think happens if that AI starts lying about what units are available or starts returning bad data? Palantir also mentions wiring this into autonomous workflows. What happens when someone prompt injects a military AI that's capable of executing workflows autonomously?
This is kind of a weird comment to be honest. I want to make sure I understand, is your assertion that prompt injection isn't a big deal because no one will wire an LLM into a serious application? Because I feel like even cursory browsing on HN right now should be enough to prove that tech companies are looking into using LLMs as autonomous agents.
Of course the application-infrastructure might be vulnerable as well in case the user IS the attacker, but it's more difficult to imagine concrete examples at this point, at least for me.
You're confusing prompt injection with jailbreaking. The danger of prompt injection is that when your GPT tool processes 3rd-party text, someone else reprograms its instructions and causes it to attack you or abuse the privileges you've given it in some way.
> spreading basically FUD around GPT
My impression is that Simon is extremely bullish on GPT and regularly writes positively about it. The one negative that Simon (very correctly) points out is that GPT is vulnerable to prompt injection and that this is a very serious problem with no known solution that limits applications.
If that counts as FUD, then... I don't know what to say to that.
If anything, prompt injection isn't getting hammered hard enough. Look at the replies to this article; they're filled with people asking the same questions that have been answered over and over again, even questions that are answered in the linked presentation itself. People don't understand the risks, and they don't understand the scope of the problem, and given that we're seeing LLMs wired up to military applications now, it seems worthwhile to try and educate people in the tech sector about the risks.