You could have a token that would work exactly as credit card. Only one token would be authorized. You cannot copy it so if you lose it, you know you lost it can be "disabled". You would then need a new one with the same key however. That key could be stored in real bank's deposit box. Hardware of the token would only allow to store the key and not read it. I mean, that's a possibility for the paranoid ones who are not happy with cell phone auth for example.

I know there's no infrastructure in place (yet) but we're talking about some kind of future, not about today. It's certainly possible though. Even secured.