Oddly, this reminds me of my mild fear of driving. I can't ignore the fact that at any moment while driving, I or anyone driving near me could apply a few paltry newtons of force in a way that causes massive injury and destruction, changing or ending lives. Every time I get behind the wheel, I have to wonder at how casually I'm about to pilot a huge mass at lethal speeds around hundreds of other people casually doing the same thing. There's nothing I can directly do to affect the mistakes of others, but I accept the risk of freak accident and resolve to do what I can to at least reduce the risk of disaster (drive defensively). Analogously, we all accept the (demonstrably small) risk of annihilation by, e.g., gamma-ray burst, and go about our lives.
It is a very useful illusion though. I take it as a sort of veneer that the brain hoists over our eyes to make day to day life bearable. There is some truth to the saying "Ignorance is a bliss".