You wouldn't hit a TPM every time you wanted a random number. Your OS would just use it when it wanted to seed its entropy pool.

This is particularly useful at boot when entropy is very low.