a stronger version of this IS the norm in many countries in Europe and large security conscious enterprises in the united states: an ID card issued that is also a high security module. it's better than your scheme because the ID card is also a computer that can respond to signing requests by a host computer and the private key doesn't leave the ID card, ever. governments issue and service tens to hundreds of millions of these cards a year.
This may be better than using one password for everything, but it's certainly not a panacea.
Edit: Or maybe have two devices: one dumb flash storage and a device that actually computes your key. You press the "compute my key" button while the storage is connected to the keychain, then disconnect it from the keychain and connect it to the pc. Next time you connect it to the keychain, it's wiped before use and never ever read.
Aren't you describing Yubikeys? Which aren't widely supported, but IIRC are still supported by some websites and apps.
It's what the FSF Europe uses for their OpenPGP cards, that you get if you join their Fellowship.
EDIT: apparently, the programmable version doesn't come with RSA support :|
I was hoping this will go away in my lifetime.
Uhm, yes, but why not use something a little more convenient, like a smartwatch with NFC/wifi?
This could easily become the norm, and should. It requires only one key for all of us, it is perfectly secure, doesn't require storing secrets on the remote end, never needs to be changed, can't be forgotten because it doesn't need to be memorized (there is one password for the LUKS encryption). There is no penalty for using the same key in thousands of places.
The problem is, no website or browser supports this kind of authentication, and most people are not savvy enough to build this sort of thing for themselves. These are both problems that should be solved; we desperately need to move the Internet away from passwords, password resets, and so on.