Actually the domain (handle) resolves to your DID which is you primary identifier and which contains your public key and without coresponding private key, nobody would be able to happily continue as you... See: https://atproto.com/guides/identity. It would be better though if the DID didn't contain the link back to the domain, so that you could continue with your identity under different domain...