News for hackers. Stuff that matters.
Re:Peak LLM by simonw on Sunday April 16, 2023 @06:37PM (#35594862)
That's exactly my problem.

Yes, it's better. Bet better isn't good enough.

When I'm building secure software, I want to know that a known exploit has been fully mitigated.

None of the software I ship is vulnerable to SQL injections, or XSS attacks, or CSRF - because I understand those vulnerabilities, and take reliable measures against them.

If someone finds an exploit, I can fix it.

With LLMs and prompt injection I don't get that confidence. If someone finds an exploit I can try and patch it with yet more pleading in my prompt, but I'm forever just guessing at what the fixes are. I can never be certain that a new exploit isn't one more layer of cunning natural-language prompting away.

That's a horrible way to build software.

Re:Peak LLM by M4v3R on Monday April 17, 2023 @02:05AM (#35597298)
I agree, but then again I don’t think prompt injection attacks are as severe as SQLi or XSS attacks. The latter can be disastrous for your application if even one is found, while for prompt injection the worst can happen is that the user will spoil their own user experience when using an LLM-based product. Of course everything depends on the use case and thus in the current stage of LLMs I would not use them in any security-critical applications.
Re:Peak LLM by simonw on Monday April 17, 2023 @02:19AM (#35597356)
That depends on what additional capabilities and tools you've made available to your LLM.

If you've granted it access to private data or given it the ability to write and execute code - both things people are starting to actually do - it could be very serious indeed: https://simonwillison.net/2023/Apr/14/worst-that-can-happen/