My point is that whether other mathematicians care or not is completely irrelevant and doesn't subtract from mathematics' power of predicting phenomena in the real world.
Each and every mathematical theory has to be consistent with basic rules of reality - if nothing else, symbolic manipulation relies on basic arithmetic and set theory. Without symbolic manipulation, you can't even express all those "abstract" mathematics - to say that "abstract" mathematics can not have correspondence to the real world is completely false, because of this basic connection.
Now that I think about it - claiming that "mathematics is a social consensus" is exactly what I'd expect from a mathematics professor - a person whose whole life is isolated from reality, and whose whole existence depends on other people caring.
There are also ways to philosophize about the nature of things in order to frame math as a human construct. I think both views can be simultaneously correct
I suspect an engineering professor would be even more compelled to highlight how social consensus is the foundation of everything else that follows. You only have to read the surface layers of this or that performance debate to see how it is unfortunately the case. "Performance" is a fluid term that doesn't mean much of anything on its own -- and people arguing that they've juiced another drop of performance juice from this or that application's fruit are often just talking past each other in terms of priorities or perspective.
The symbolic manipulation at the heart of mathematics is a byproduct of language -- another system beholden to social consensus. It's inescapable.
That's only a tiny fraction of what math does. An interesting and useful one, and mathematicians have out a lot of work into studying basic arithmetic. They have expanded out into numerous other forms, some of which turn out to have correspondence to the real world like non-Euclidean geometry.
Others turn out to be completely abstract and are merely curiosities. There are an infinite number of them, each containing truths, almost all of them of no interest. Interest is defined by mathematicians, not physics. Even so it turns out to sometimes be useful, such as the beautiful theorems of prime numbers that drive Internet security centuries after they were invented.
That is what the OP means. You can make up any axioms you want and prove true theorems. But the hard part is convincing other mathematicians to care.