At that point, they can regenerate the key-pair and have the SSA official sign the public key and keep that on file.
Now, presumably, it is possible to forge both documents, but I would think that the government could check their records (federal and state) to verify the authenticity of the provided documentation.
This would take a lot of solid investment politically, and technically, to make it even slightly feasible to work for the average Joe.
Resetting it outside the normal certificate expiration time would require that one go to the local branch of the government office to do so (much like getting a replacement social security card or replacing a lost or stolen passport). At that point, you would have to provide proof of your identify that would be verified by the government agency.
There's no reason not to keep SSN as an identifier. Just the same as I wouldn't change my name if I suffered identity theft. Instead there needs to be authentication (eg via a method such as a token, 2FA or whatever) - and it's that which needs to be resettable.
Simply making use of SSN alone illegal in certain industries would be a reasonable approach: it would stop current problems whilst not insensing the "mark of the beast" brigade.