Short answer: not every physicist works in a lab. Theoretical & mathematical physics are entirely about working with mathematical models of phenomena that other people observed in a lab. It's enough to understand that any theory is rooted in the experimental, and should be falsifiable by it.