Even so, one who actually possessed that private key could sign a new message with it, and we could use the public key we know to verify that signature.