If people have to go through a lot of trouble, I suspect they would quickly learn a way to minimize the risk of having to do so. I suspect that most people are careful with their passports so that they don't lose them, because it is a bit of a hassle to get a replacement.
1. The use of a physical authentication mechanism makes exfiltration far more difficult. The problem with passwords is that they can be sniffed with absolutely no awareness by the victim. A device or token you have to have yourself is going to make its absence known.
(This presumes it's not cloneable.)
2. That reduces the attack surface for phishing. What I suspect is is that you'd see other forms of attack -- say, trying to get recurring charges inserted into bills or some such, rather than full account access.
3. In the event of token compromise, the ability to determine after the fact that a device was compromised and invalidate the subsequent transactions would be helpful. (Sorting out who bears the risk for that would be an interesting question. I'd like to put the burden on the banks, as they're best positioned to detect such activity.)
Gibberish is not something humans can remember. Technical people use a password manage, but that just limits us to one password that we use often enough that we have a chance to remember it. One wrong knock to the head though and all the passwords stored in the manager are gone - unless you wrote the password down someplace, but then that place is your sock drawer: probably not secure enough.
By that I mean that we need to educate the masses that a few letters and numbers are as valuable and powerful as the combination to their home safe or keys to their car. People just aren't inclined to exercise the same degree of caution over some gibberish letters as they are the cash in their sock drawer, even though the gibberish is potentially much more valuable.
I'm skeptical that such an education is possible without it being ingrained from youth and even then it may be impossible to achieve a high enough degree of understanding to make the kind of shift you envision.