Category theory isn't really a replacement for set theory any more than order theory is a replacement for set theory. What is important is not what your "foundational system" is, but that just like there are many concepts we learn in the context of set theory but are just very generally useful (equivalence classes, cardinality, injective/surjective functions), there are also similarly very useful concepts in category theory like adjoint functors, limits and universal mapping properties.
For analysis, Peter Scholze and Dustin Clausen would disagree:
> https://en.wikipedia.org/wiki/Condensed_mathematics
To quote from https://www.math.uni-bonn.de/people/scholze/Analytic.pdf linked there:
"Mumford writes in Curves and their Jacobians: “[Algebraic geometry] seems to have acquired the reputation of being esoteric, exclusive, and very abstract, with adherents who are secretly plotting to take over all the rest of mathematics. In one respect this last point is accurate.”
For some reason, this secret plot has so far stopped short of taking over analysis. The goal of this course is to launch a new attack, turning functional analysis into a branch of commutative algebra, and various types of analytic geometry (like manifolds) into algebraic geometry. Whether this will make these subjects equally esoteric will be left to the reader’s judgement.
[...]
The author always had the impression that the highly categorical techniques of algebraic geometry could not possibly be applied in analytic situations; and certainly not over the real numbers. The goal of this course is to correct this impression."
Spoken like a true algebraist.
For anything related to stats, PDEs, and optimization (basically that subset if mathematics that is most useful to other sciences), category theory is a horrible foundation.
While it seems you can recover (with a looot of work) some existing theory, no sane researcher in these fields uses category theory.
And there is also no motivation to do so, since unlike for algebraic topology, algebra etc. the categorical viewpoint doesn't really make it clearer, what, e.g., the weak solution to a PDEs is, and why it coincidences (or not) with the classical solution.