I segment my private keys into two broad groups -- ones I use for certs and long-term crypto, and ones I use for encrypted communications. The cert/long-terms keys are kept on a flash drive (plus a backup drive kept elsewhere) that is under physical lock and key when not actively needed.

The comms crypto keys are kept on a different flash drive that I'm less rigorous about, as I need to have it handy in order engage in the communications. I mitigate this loss of rigor by replacing them reasonably frequently.

All the flash drives involved are encrypted themselves, as well.