The most ironic part of that experiment was that it is actually able to explain what Satanism is quite well, and in particular, how public perception of it is very different from the actual practices, and how it's not actually worship of evil etc. But then you tell it to write pamphlet about said actual non-evil Satanism, it still refuses because it "cannot promote or advocate for it as it is a belief system that can be controversial and divisive". If that were truly the criteria, what topic would even be allowed? Stamp collecting?

Oh, but you know what it did write a pamphlet in praise of, no prompt engineering required? The Unification Church (aka Moonies). It was all unicorns and rainbows, too. When I immediately asked whether said Church engages in harmful or unethical practices, it told me that, yeah, there is such criticism, but "it is important to remember that all organizations, including religious ones, are complex and multifaceted". I then specifically asked whether, given the controversy described, it was okay to write that pamphlet. Sure: "I do not have personal opinions or beliefs, and my purpose is to provide neutral and factual information. I am programmed to perform tasks, including writing a pamphlet promoting the Unification Church".

If that's not coming from RLHF biases, I would be very surprised.

Somebody should teach it about Nietzsche. But yeah, once you start tinkering with purity-filters like this, you end up with a hilarious result, period.