A lot of contemporary mathematics have physical roots. Some things I like: Donaldson theory and Seiberg-Witten theory in differential topology, Gromov-Witten theory in sympletic/algebraic geometry. Knowing physics is not necessary to learn these things, but some rough idea of what QFT is doesn't hurt.