Good point. Why do we have passwords anyway? Just to authenticate at unanticipated access points? We could always just do two-factor authentication for that. A code would be sent to Google Authenticator on your phone or something like that, encrypted with your public key, and you'd just decrypt it with your private key. The phone would be secured with your password or fingerprint, which is never sent anywhere. The code would allow you to auth with the new access point. If the cellphone was connected to the internet, it could send the key directly and the site would log you in. If not connected to the net, it could use bluetooth or optical or sound to communicate with the access point (eg web browser running on a computer) to send the code. Only in the worst case would you have to type anything in yourself.

Since most people walk around with their cellphones, you could even precompute a couple keys to unlock a site, for those times when there is no internet connection. Of course, your phone should be secured with a password and you should be able to revoke the keys if anything gets lost.