I think what grandparent and others are saying is that you also revoke the secrets contained in the ciphertext, e.g. if it's an AWS key you would revoke that AWS key on AWS's side as well as encrypting new secrets with a different encryption key.

Obviously this is a huge hassle and isn't easily done with all kinds of secrets (which is what I think you're getting at?). But it's also often necessary.