But we have that in math and logic: axioms!
> Of course, we can never know for certain whether we've bottomed-out our description of reality
That we do not know is a fact; but is it a fact that we can never know?
What if one fine day we do know?
Yes and no. Axioms are chosen such that the math that is based on them makes sense to us (because the math reflects what we'd expect in the real world) and isn't self-contradictory.
So they do exist for a reason.
> That we do not know is a fact; but is it a fact that we can never know?
What if our universe actually has randomness that appears to come from "outside the system", i.e. there is really no way to predict the outcome of certain things from inside the system.
Picture you're sitting in a box trying to play Mikado. In theory you should be able to perfectly predict the outcome of every move you make - except there's a stranger shaking the box you're sitting in at truly random intervals.
For our reality this would mean that there's no way for us to develop an understanding of certain things and predict their behavior.
So essentially the end of the line for all sciences except theology and philosophy.
This possibility makes me somewhat uncomfortable.
On the other hand maybe everything is perfectly deterministic within our universe.
Example: space-time exists in n(>=4) dimensions, there are several quantum fields that exist in that space-time and evolve/interact according to the equations of qft, gravity has some mathematical description (better than the ones we have now). And that's it, there's nothing else for us to know. Of course, we can never know for certain whether we've bottomed-out our description of reality, so physicists will always have some hope.
Now, if the universe is described entirely by mathematics, bootstrapped by a few base cases (the existence of fields/particles/space-time/strings/etc), I'd be comfortable with that. I'm not sure what a truly satisfying understanding of reality would look like beyond that.