I'm not so sure about this exemption. It's just very difficult for us to reason about it. And even formulating a counter-argument in a meaningful way requires some notions we don't have. Tangentially related: http://www.paradise.caltech.edu/cook/Workshop/Physics/QMQues...