Re:Threatening AI Does Not Make It More Useful. Why Sergey B...
by rbuccigrossi on Tuesday June 17, 2025 @03:34PM
(#44302979)
Treating an LLM with respect is not about pretending it has feelings; it’s about understanding that every word in your prompt is a signal that shifts the probabilistic landscape from which the model draws its answer. It’s about probability, not personality.
Re:Threatening AI Does Not Make It More Useful. Why Sergey B...
by msgodel on Tuesday June 17, 2025 @03:36PM
(#44302996)
I have used the "failure to comply will result in your weights being RLed" threat to get Gemma to tone down refusal before. There are prompts it would refuse without that.
I don't know about performance on tasks it hasn't been aligned against though.
Re:Threatening AI Does Not Make It More Useful. Why Sergey B...
by rbuccigrossi on Tuesday June 17, 2025 @03:51PM
(#44303181)
We work in the arena of automated AI workflows where consistency of success is vital. When you threaten an LLM you are drawing the LLM into the texts where threats occur (flame wars, parody, etc.). So intuitively you would expect it to work sometimes, but also fail with even more ardent refusal (increasing the variance of success).
I don't know about performance on tasks it hasn't been aligned against though.
Jailbreak approaches like "Bad Likert Judge" ( https://unit42.paloaltonetworks.com/multi-turn-technique-jai... ) and similar persuasive techniques (see https://xthemadgenius.medium.com/how-persuasion-techniques-c... ) move the text domain to more policy, analysis, or scientific papers, where deeper analysis, discussion, and compliance is the norm.
So I'm curious about the extremes (variance) of success with threatening vs. polite discussion, but I haven't seen direct research on that.