As a side note: for example we had some backups that are probably useless, because they are way too small. Catching this would mean more manual regular checks, or some automated rules, at which point it becomes quickly more complex again.
Storytime: I did not check the restoration of my backups some time ago and had a faulty harddrive, so I needed to restore the backup. Backup was also as dumb as possible - essentially a tar and encrypting it with openssl. So I reinstalled the server, tried to decrypt it - got the error, that the key was wrong. Took me a good weekend to find out, that openssl changed the default hash algorithm between openssl 1.0 and 1.1. This would not have been catched with the proposed system, but now I really pin all default options in my scripts as well.
have you ever tried typing a private key from a piece of paper? once i was in a similar situation and gave up and just drove to the colo.
$ entropy 32|zbase32-encode
pu3zrux6t6cqrmmyesdxtppxiudxjcndrx3bomjuyaupa61493no
$ entropy 32|phrase-encode
afar-pimple-unwind-imagine-buckets-today-duke-sober-dehydrate-rebel-online-nudged-bamboo-saxophone-eluded-tattoo-pause-bays-ungainly-tasked-jingle-topic-null-enragedOpenSSH private keys are armoured by default, gpg-keys can be exported and imported in an armored format - and everything else can be just printed as hex representation with whatever tool (e.g. `od -Ax <file>` or any other).
fd3223ec 20f55ae7 6fddc979 d41e2276 25255516 b08f5cd4 3d66d676 a054d2bb
Have an offline backup printed along with the disaster recovery checklist and documentation and put them in a safe in your company - the checklist should be dumb enough that your drunk self can use it at four in the morning, because you were the nearest employee when everything went down.
Ensure that you have stupid manual processes in place on rotation of the safe's PIN and encryption keys in general, including a sanity check if the newly generated keys actually work (e.g. if they are used for your backup storage, actually back something up and restore it). Ensure that the safe's PIN is available to at least another person and used regularly (e.g. because you store your backup tapes there).
If you feel that you need to change from this very simple system to a more complex one, ask yourself why. What does your change actually add in terms of security and what risks does it add.
In the end, you want your system available to customers and the security you add is to not only secure the data, but actually to know who can access it (the auditing part).