So what about large asteroid strike levels of improbability? I have to reject your premise that there is no probability where you can stop worrying about the problem.

(This might be a moot point, because I'm not sure current methods can ever get to this level of accuracy, due to limitations of the training data. Needs an entirely new method or a clever insight to optimize for truthfulness with unclean training data, and InstructGPT hasn't made much progress on this, and it might not even be possible)