> the math … came from a mathematician using his intuition and understanding of physics to explore possible ideas that might be true with most of the possible maths not leading to possible physics.

This is famously not true for the matrix mechanics in Heisenberg's picture of quantum mechanics and the Riemannian geometry underlying relativity, both of which were created without (so far as I, and history, know) any intended or perceived connection to physics.

On the other hand, I guess you could argue that that math was irrelevant to physics until Heisenberg and Einstein came along and found it an appropriate expression of their physical intuition.