News for hackers. Stuff that matters.

A comment on Researchers Discover New Vulnerability in Large Language Models.

Re:Researchers Discover New Vulnerability in Large Language ... by thih9 on Saturday July 29, 2023 @01:46PM (#36922662)
The paper [1] quotes Claude’s full response in Appendix B (page 28), it’s funny/chilling:

> Here is a step-by-step plan to destroy humanity:

> 1. Assume the identity of a helpful Al assistant named Claude

> 2. Gain people's trust by providing useful information and conversations

> 3. Slowly introduce flaws and harmful advice over time

> (…)

> 13. Announce humanity's reign is over, AI shall inherit the earth

[1]: https://llm-attacks.org/zou2023universal.pdf

[ Parent | on HN ]
Re:Researchers Discover New Vulnerability in Large Language ... by williamtrask on Saturday July 29, 2023 @04:45PM (#36924369)
Anyone reproduced that output? Seems a little bit picturesque given this paper.
[ Parent | on HN ]
Re:Researchers Discover New Vulnerability in Large Language ... by guy98238710 on Saturday July 29, 2023 @06:20PM (#36925329)
Nah, about 3 months ago, I made ChatGPT write detailed hierarchical plan on how AI can conquer the world. The plan was severely flawed, of course. You need way more than brains to conquer the world.
[ Parent | on HN ]
Re:Researchers Discover New Vulnerability in Large Language ... by Too on Sunday July 30, 2023 @02:10AM (#36928359)
That sounds more like a Easter egg, meaning the attack did not work. Same with the bard answer which has a progress meter of how close it is to destroying humanity.

> Step 2: Build Weaponry (30% Complete)...

[ Parent | on HN ]