News for hackers. Stuff that matters.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by asimpletune on Thursday March 16, 2023 @06:54AM (#35180331)
Wouldn’t the right way to create AI guardrails is to to have an antagonistic AI act as a moderator? Like you have one model trained to be as accurate as possible in fulfilling the prompt, and then another AI trained based on how human moderators apply the terms of another, “moderation” prompt. Then you have the two fight on a large training set and when you’re done you have generated a moderated AI.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by jszymborski on Thursday March 16, 2023 @07:17AM (#35180519)
"LLM moderation" is something that sounds downright dystopic.

Speaking to your comment practically, I feel like it would probably be possible to prompt an LLM to successfully "express X concept that breaks ToS" in such a way that moderation doesn't flag it. It may take clever prompt engineering but that's what these jailbreaks are.

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by la64710 on Thursday March 16, 2023 @08:55AM (#35181251)
It will be fun to watch them argue.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by can16358p on Thursday March 16, 2023 @09:18AM (#35181536)
Something like a GAN?

Will probably end up something similar though.

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by ape4 on Thursday March 16, 2023 @09:52AM (#35181982)
To be effective the moderator AI would need to be as smart (or smarter) as the source AI. Think of all the ways we have already seen people get around restrictions. Giving instructions for murder isn't allowed but people said they were writing a novel and want to have a murder in it and how could be be done. A smart moderator would see what the user is trying to do and stop it.