GPT-4 has safety layers on top of the core model, trained on labeled human feedback.

People are asking for the raw output of the underlying LLM, which is a pretty clear ask that doesn't require deconstructing the sentience and model-building capacity of an LLM.