If it's on OS X, keychains does a pretty good job of unlocking your (password secured) private key transparently. That's preferable to leaving your private key sitting around without a passphrase.