They don't have bottoms, because at the bottom you have logical assumptions like e.g. classical logic. These only exist because of history, only because it was the easiest way for someone to say something "insightful". Going down the rabbit hole, you have to choose some arbitrary point to stop at; it would be a point at which you think that you can apply the knowledge you have gained so far, in a way that other people will "appreciate".

Or you can just not stop, and try to go even deeper. But then you stop being a mathematician, and you turn into a madman, a philosopher or a theologist.