I'm sorry but even if this is objectively valid, this not a claim that you can support. There are entire sub-fields of maths in which there are no known physical attachments.
Now, that's not to say that some time in the future we won't discover the relationship between every mathematical concept and some physical system. But at this moment, the claim you are making has a numerable set of counter examples, with a very large order. For one example, take the Banach–Tarski paradox:
>Given a solid ball in 3‑dimensional space, there exists a decomposition of the ball into a finite number of non-overlapping pieces, which can then be put back together in a different way to yield two identical copies of the original ball.[1]
This is, as far as we know, physically nonsense.
That being said, there is quite a lively debate among mathematicians as to whether or not math is tied to reality.
[1] http://en.wikipedia.org/wiki/Banach%E2%80%93Tarski_paradox
Regardless of how abstract you get, how far you go, math is always tied to our physical reality. The basic operations are reflections on properties of our universe. The "kind of stuff" mathematicians do" allows us to model and reason about our world in ways that wouldn't be possible any other way (that we know of).