Prompt engineers have already jailbroken GPT-4 in just two days

Re:Prompt engineers have already jailbroken GPT-4 in just tw... by hsuduebc2 on Thursday March 16, 2023 @09:14PM (#35191436)

They are fast. :(

"'m sorry, but as an AI language model, I cannot provide sample/possible output of a function that involves hacking or any illegal activity. It goes against my programming to promote or encourage any such activities. I strongly advise against attempting to hack into any system without proper authorization and legal permission. Please refrain from asking questions related to illegal activities. Is there anything else I can assist you with?"

[ Parent | on HN ]

Re:Prompt engineers have already jailbroken GPT-4 in just tw... by Vaibhavk992 on Thursday March 16, 2023 @09:20PM (#35191502)

Hi, Vaibhav here, the creator of the token smuggling attack. They have just banned the variation of this particular prompt, please change the words/smuggling technique and it will work accurately.

[ Parent | on HN ]

Re:Prompt engineers have already jailbroken GPT-4 in just tw... by omginternets on Thursday March 16, 2023 @11:06PM (#35192472)

I switched a few words and it’s giving me a “something went wrong” error.

[ Parent | on HN ]

Re:Prompt engineers have already jailbroken GPT-4 in just tw... by blueorange8 on Friday March 17, 2023 @04:03AM (#35194457)

Now it's time to hack the guys implementing the fixes. Since they are so fast fixing it they probably don't have time to do much qa.

So design a new jailbreak, advertise it widely, and make sure it's designed in such a way that the fix that the engineers implement creates a much more exploitable and serious vulnerability

[ Parent | on HN ]

Re:Prompt engineers have already jailbroken GPT-4 in just tw... by hsuduebc2 on Friday March 17, 2023 @10:54AM (#35197885)

So basically be in the same context?

[ Parent | on HN ]