Most modern phones have secure elements that can generate and store a private key that cannot be extracted through software. Also, it is easy to set up things such that a physical confirmation is required to sign something (this is eg. what some U2F keys or touch ID do).

Of course, you still need a procedure to map public keys to identities and people need to secure their phones in order to prevent someone stealing the phone to make signatures.

But using the secure element of a phone for various forms of authentication is orders of magnitude safer than relying on a credit card or social security number.

The "only" other problem that will remain is that you will need a secure supply-chain, otherwise this will happen:

http://www.zdnet.com/article/id-card-security-spain-is-facin...

https://www.reuters.com/article/us-gemalto-cybercrime/hack-g...

If you do something like this you actually need to be serious about it and establish a rigorous vetting/auditing process -- not just hand the contract to whoever donated the most to your election campaign.

Maybe set-up a 3-year long NIST competition or something, like they do when choosing new crypto standards, and establish the winner this way.

The other side of the equation, allowing services to interface with these cards securely, is already being solved by the FIDO 2.0 spec.

Standards like FIDO and others allow for browsers and websites to utilize these cryptographic functions of smart cards in ways to handle website authentication. The technology exists for smartcards to handle authentication, it is simply a matter of us moving to this technology.

USB smart card readers are fairly cheap. Many business-focused laptops have been available with smart card reader options for many years.