Except this isn’t really true in the common sense meaning (schoolchildren do arithmetic fine without knowing about category theory) or in the formal sense you’re trying to get at (there are formal axiomatic foundations which arithmetic can be based on which do not need category theory. A simple proof is by causality: arithmetic was successfully formalised before category theory was invented)

You should read the famous book by Linderholm, Mathematics Made Difficult