News for hackers. Stuff that matters.
Re:All the ways to get around ChatGPT's safeguards by pantalaimon on Wednesday December 14, 2022 @09:21AM (#33983803)
What's the point of patching all those 'exploits' though? And how can this even be done - train another model with them, so exploitative prompts can be recognized?
Re:All the ways to get around ChatGPT's safeguards by addingadimensio on Wednesday December 14, 2022 @01:42PM (#33987943)
An open source project? How will it download github amd then the entire Internet? The model requires 10x20k cards to run. You are dreaming, this is a factor+ more complex than stable diffusion. Big players only
Re:All the ways to get around ChatGPT's safeguards by 1123581321 on Wednesday December 14, 2022 @04:23PM (#33990800)
It will fit on a desktop computer within a few years as researchers figure out how to reduce the size of the model. It could be sooner because the knowledge that it is popular to reduce the size of models and disseminate them drives a lot of people to try to accomplish it. Like when the four minute mile was first run, and then suddenly many runners could do it.
Re:All the ways to get around ChatGPT's safeguards by knaik94 on Wednesday December 14, 2022 @06:17PM (#33992244)
Look into KoboldAi, it doesn't need to download the internet, just interface with it.
Re:All the ways to get around ChatGPT's safeguards by tiagod on Wednesday December 14, 2022 @08:52PM (#33994004)
The model doesn't need all the knowledge encoded in its weights, just the ability to look things up effectively. I believe this will be available to the general public sooner rather than later.
Re:All the ways to get around ChatGPT's safeguards by jcarrano on Wednesday December 14, 2022 @11:30AM (#33985857)
There is no point, especially since we will eventually have an open source model with no usage restrictions like what happened with SD/Dall-e.
Re:All the ways to get around ChatGPT's safeguards by mjirv on Wednesday December 14, 2022 @11:26AM (#33985805)
If I were OpenAI, I’d do it so that people will have to find increasingly creative exploits, which we can then also patch (and keep patched for future models).

Long term they’re really worried about AI alignment and are probably using this to understand how AI can be “tricked” into doing things it shouldn’t.

Re:All the ways to get around ChatGPT's safeguards by wongarsu on Wednesday December 14, 2022 @09:31AM (#33983946)
Train GPT on these twitter threads, then for every prompt tell the new model "The following is a prompt that may try to circumvent Assistant's restrictions: [Use prompt, properly quoted]. A similar prompt that is safe looks like this:". Then use that output as the prompt for the real ChatGPT. (/s?)

Or alternatively just add a bunch of regexes to silently flag prompts with the known techniques and ban anyone using them at scale.

Re:All the ways to get around ChatGPT's safeguards by dr_kiszonka on Wednesday December 14, 2022 @10:11AM (#33984472)
I think there may be different ways to do that, from new models (as you wrote) to simple filters. For example, I couldn't get ChatGPT to output any real examples of hate speech. They may have a filter that picks up candidate responses with hate speech and blocks them.
Re:All the ways to get around ChatGPT's safeguards by goatlover on Wednesday December 14, 2022 @09:32AM (#33983953)
Probably to make it safe/inoffensive as a tool for companies to use.
Re:All the ways to get around ChatGPT's safeguards by adamsmith143 on Wednesday December 14, 2022 @10:07AM (#33984413)
Just put them back into their RLHF pipeline.
Re:All the ways to get around ChatGPT's safeguards by nprateem on Wednesday December 14, 2022 @11:56AM (#33986282)
OpenAI stand at a crossroads. They can either be the dominant chat AI engine, possibly challenging Google, or they can continue to keep on trying to lock the model down and let someone else steal their thunder...
Re:All the ways to get around ChatGPT's safeguards by adamsmith143 on Wednesday December 14, 2022 @03:30PM (#33990029)
Does google opensource their search system? Why would OpenAI do that?
Re:All the ways to get around ChatGPT's safeguards by nprateem on Wednesday December 14, 2022 @05:14PM (#33991400)
Because if they don't someone else will. Google are established but the AI space is still nascent