It's likely the most probable continuation.

If answer != google answer then censored

is the wrong way to think about an LLM.

What is the process by which you believe such a "moral directive" was baked in?

Put differently, if the model is simply trained on a large corpus of free-form web text, and naturally learns that the most likely reply to "how do I do X?" is "X is dangerous, don't do it", is that baked-in censorship? Or do you envision that the training process included special guard rails outside of the raw training data?

> Or do you envision that the training process included special guard rails outside of the raw training data?

I imagine they mean this, or that the training data included a lot of synthesized exchanges to that effect.

I personally can't imagine that, using real, human-written text corpora on the Internet and otherwise, the most likely continuation for "how do I do X?" is "X is dangerous, don't do it". Usually people don't mention a topic only to shut it down - they either cover it properly, or don't mention it at all.

A quick Google for similar phrases brings up quite a few examples. Stackoverflow also has lots of answers like that. It's not super common, but I wouldn't be surprised if that got generalised to some answers.