Re:GPT-OSS Reinforcement Learning
by BoorishBears on Saturday September 27, 2025 @04:35AM
(#45394110)
Use constrained generation
Re:GPT-OSS Reinforcement Learning
by artdigital on Saturday September 27, 2025 @06:11AM
(#45394528)
Mind explaining?
Curious as well
Re:GPT-OSS Reinforcement Learning
by BoorishBears on Saturday September 27, 2025 @03:05PM
(#45398538)
If you constrain the model to a JSON schema, most frivolous refusals go away.
And if you finetune on a few formatted examples the effect is even greater
Do you mean like structured outputs? Unfortunately here the model is guided to explicitly tell you when you violate the rules and why, it can confuse it's system rules with the game rules and say you're not allowed to ask a question about copyrighted material etc.
Re:GPT-OSS Reinforcement Learning
by artdigital on Saturday September 27, 2025 @04:55AM
(#45394211)
Try the uncensored/jailbroken variants like openai-gpt-oss-20b-abliterated-uncensored-neo-imatrix
I just tried to ask it how to make crystal meth and it generated a very detailed step by step guide
I have heard that uncensorted gpt-oss is not very good because of it being trained mainly on synthetic data. Is not not true?
Iirc abliteration (ablation?) can be done without "training" and is pretty quick. It finds the individual weights related to the concept you want to ablate, and modifies those weights to "deactivate" them. Precision brain surgery, to anthropomorphize.
The problem with synthetic data would be that the censored information would not be in the training data at all.
1. https://guessix.com/