Re:Heretic: Automatic censorship removal for language models
by Qwuke on Sunday November 16, 2025 @11:41AM
(#45946393)
I've seen Optuna used with some of the prompt optimization frameworks lately, where it's a really great fit and has yielded much better results than the "hyperparameter" tuning I had attempted myself. I can't stop mentioning how awesome a piece of software it is.
Also, I'm eager to see how well gpt-oss-120b gets uncensored if it really was using the phi-5 approach, since that seems fundamentally difficult given the training.
Re:Heretic: Automatic censorship removal for language models
by p-e-w on Sunday November 16, 2025 @12:08PM
(#45946603)
FWIW, I already used Heretic to decensor gpt-oss-20b [1], and it works just fine. Note that the number of refusals listed on the model card is actually an overestimate because refusal trigger words occur in the CoT, even though the model doesn't actually end up refusing in the end.
Re:Heretic: Automatic censorship removal for language models
by NitpickLawyer on Sunday November 16, 2025 @12:23PM
(#45946733)
What's your intuition on other "directions"? Have you tried it on something other than "refusals"? Say "correctness" in math or something like that. I have some datasets prepared for DPO on "thinking" traces that are correct / incorrect, wondering if it'd be something that could work, or if it's out of scope (i.e. correctness is not a single direction, like refusal training)
Re:Heretic: Automatic censorship removal for language models
by p-e-w on Sunday November 16, 2025 @12:39PM
(#45946858)
The problem is that in order to do optimization, you need a classifier that can distinguish the two types of responses (like refusal/compliance). In case of refusals, that's relatively easy to do using trigger words like "disallowed" or "I can't". I imagine this would be much, much harder to do automatically for classes like correctness.
And I also suspect, as you hint at, that "correctness" isn't just a direction in residual space, but a concept so broad that no simple mechanistic description can capture it.
Re:Heretic: Automatic censorship removal for language models
by p-e-w on Sunday November 16, 2025 @12:11PM
(#45946623)
Please let me know if you encounter any problems with the 120b! I'm really interested in how well it will work. When presented with the Pareto front at the end, I recommend choosing a configuration with a KL divergence below 1, even if the refusal rate seems high. The gpt-oss models are trained to do an internal monologue about refusing in the CoT, so the actual refusal rate is often substantially lower because Heretic's refusal classifier gets confused by the trigger words.
Nicely done to pair that with something as fun as censorship removal, currently in the process on running it on gpt-oss-120b, eager to see the results :) I'm glad that someone seems to be starting to take the whole "lobotimization" that happens with the other processes seriously.