Part of it is that exact mathematics run afoul of several atheist articles of faith. Depending on your interpretation it either disproves the "Jacobian" scientific model, or more flexibly interpreted it means science is at best a subset of what governs the universe (assuming -yes, assuming- it isn't entirely wrong). Godel's theorem runs afoul of the principle that science can explain everything. It answers the question whether mathematics directs and/or predicts the events in our universe : it doesn't, it can't. It's too "weak" a theory.

We're at a very exiting point in time where we know basic mathematics to be wrong. It predicts a great many things, but there's problems that make it thoroughly unsatisfactory. In a way it's like physics in the beginning of the 20th century, with the black body radiation problem. Of course, we have been in this less-than-satisfactory state for ~70-80 years now, and no Einstein in sight ...

Godel's theorem means that there's an infinite set of empirical truths (ie. simple experiments you can try out with marbles and bags) that are completely unexplained by mathematics - and thus by every science built on top of it.

Worse : this is not a fixable problem. Sure we can fix it for specific problems. Wherever we see an obvious leak (say the birthday problem, or large cardinal number problems) it can be plugged with a new well-chosen (or -more often- ill-chosen, like choice) axiom, but there's infinitely many leaks and the proof means that there's no plug that will stop any significant number of them.

So right now in the set of all empirically observable events, there's a set that's explainable by science, and there's a set unexplained by science. The unexplained set is at least as large as the explainable set (and keep in mind that's because both sets have been proven to have a cardinality of at least the largest known cardinal number, given the actual definitions of those 2 sets I'd say the unexplained set is going to turn out to be bigger).