Wont that make the backups useless in case of a data loss (i.e. always)?

You don't keep a single copy of each key, but store enough redundant copies to get the proper number of nines. Preferably that's redundant geographically, in terms of storage technology, and in write frequency.

The important part is just that the keys don't end up in long term cold storage. Either it's only retained for a short period (e.g. tape backups that get rotated after two weeks), or it supports live deletion.