While the consumer will soon be irrelevant, I agree with the basic premise: neutered AI isn't helping.
At the same time, overrepresentation of evil concepts like 'Nazis are good!' or 'Slavery is the cheapest, most morally responsible use for stupid people' could lead to clear biases (ala Grok 4) that result in alignment issues.
It's not a clear-cut issue.
Hallucinations come from lack of information, or rather training data, in a particular field.
It is NOT a malicious try at feeding you untruthful answers, nor is it a result of getting trained with misinformation.
Unless you're about to fix hallucination, isn't it more harmful to have AI administer inaccurate information instead?
Refusing to answer lobotomy-related questions is hardly going to prevent human harm. If you were a doctor trying to research history or a nurse triaging a patient then misinformation or neglected training data could be even more disastrous. Why would consumers pay for a neutered product like that?