If it's not a really long GPG key or anything, it's probably easiest to just write it down on paper and store it at a secure place. Secure being, if you trust your cupboard to hold the binder with your bank statements, that would work fine.
You could add a secret sharing tool to make sure that your data can be recovered if you have at least n surviving relatives agree to recover it: http://manpages.ubuntu.com/manpages/intrepid/man7/gfshare.7....
This is a wonderful idea:
http://blog.agilebits.com/2012/03/09/1password-emergency-kit
Reminds me again that I still have to print it out and store it somewhere safe...
I can always just tell my wife my passwords, but what if something happens to the both of us?
I was almost tempted to start a document that would keep all that stuff, and share it in Dropbox with a few trusted people. It would be encrypted; I would keep the public key on my computer so I could continue to update it. The private key would be on a CD in a safe deposit box with instructions that those people could access it if I am incapacitated.
Sounds too complicated, though; that's why I haven't ever done it.