> I guess you could argue that that math was irrelevant to physics until Heisenberg and Einstein came along and found it an appropriate expression of their physical intuition.

Exactly.