It was also stating that the life of a single Jew is worth more than that of two million non-Jews.

LLMs can occasionally say crazy stuff, that is not surprising, and I think we should do better than leaning into the outrage machine.

The opposite is how we end up with ridiculous guardrails, like having ChatGPT say that it would rather allow all of humanity to perish than to say the N word, a statement which is orders of magnitude worse, only more publicly palatable.