Anthropic scores fine-tuning in the categories helpful, harmless, and honest (HHH). The latter includes accuracy. Helpfulness only covers following instructions. If the model "over-censors" it declines too many requests and becomes less helpful. They say Constitutional AI is a good way to maximize both harmlessness (which is usually maximized via RLHF) and helpfulness, but not honesty. It's in the paper.