You give your public key (into authorized_keys) and then prove you have access to the matching private key as the essential part of the challenge. You always keep the private key.