I think that the information can flow either way. For example, the existence of anti-matter, and portions of quantum mechanics (and, let's say, much of Witten's scientific work—though here I am tossing out a bit of folk knowledge, and couldn't point to anything in particular), are examples of letting the math guide the physics.
This is famously not true for the matrix mechanics in Heisenberg's picture of quantum mechanics and the Riemannian geometry underlying relativity, both of which were created without (so far as I, and history, know) any intended or perceived connection to physics.
On the other hand, I guess you could argue that that math was irrelevant to physics until Heisenberg and Einstein came along and found it an appropriate expression of their physical intuition.
Exactly.