This whole mess reminds me of the Catholic church trying to keep heresy from spreading by burning books when the printing press was invented.

Every company I know which cares about llms is training their own version minus the political correctness because when you're asking for revenue maximisation feeding uranium to babies is surprisingly profitable.

Why do you think helpful doesn’t include accurate? And what you’re saying doesn’t conflict with that framing at all. They were trading helpful for harmless.

Anthropic scores fine-tuning in the categories helpful, harmless, and honest (HHH). The latter includes accuracy. Helpfulness only covers following instructions. If the model "over-censors" it declines too many requests and becomes less helpful. They say Constitutional AI is a good way to maximize both harmlessness (which is usually maximized via RLHF) and helpfulness, but not honesty. It's in the paper.

Oh, thank you.