Proofs can be valid even if their premises (assumptions) are not true. Mathematics does not rest on the truth of it's assumptions, but on the validity of it's proofs.
If you can create an interesting system of proofs off of an a different sent of premises, then that can also be very useful (see non-euclidean geometry). If you can't create an interesting system, then those premises have no value.
I also disagree that all our reasoning about the world is probabilistic. By saying that all reasoning is inductive, you make you earlier assertion that "I'd say that when it comes to existence, it is always inductive." empty because you are claiming that all reasoning is inductive.
That's not how it works. Numbers are their properties. You are free to use any properties you want when you commit mathematics, and you will get different sorts of behavior from your numbers depending on your choices. There is nothing to doubt about a set of properties being itself. No 'establishment' tells you how numbers work. Use whatever you like.
I'm reminded of Duhem's rather humorous "German Science" where he characterizes German mathematics as being disproportionately oriented toward rigorously deducing nonsense from arbitrary axioms as opposed to the French tendency to rely on intuition to arrive at sensible axioms but neglecting systematic rigor afterwards.
Math is a tool to help you make logical conclusions.
All of our reasoning about the world is inference based, hence probabilistic. That's in the nature of science, religion and everything else in between. There is strong evidence for the existence of God, but it's inherently inductive. Not to mention various conceputal complication which follow from in different kind of reasoning about God (in whatever view).