You ask a very wise question. Unfortunately, I think it's unknownable. The best we know is that the answer is more than none and less than all. The more you get towards "all" the more prevention measures cost to implement. For instance, managing a mature backup and imaging operation at scale may be conceptually simple but is both complex in practice and far from free.
Hospitals in particular are the scene of some interesting conflicts between security and usability. There are a lot of stories about health staff doing things like jamming open medication dispensing machines so that they could get on with the job instead of dealing with security measures they experienced largely as obstacles.
Can you imagine throwing yubikeys into a scenario like that, where people already have an adversarial relationship with IT and security measures? What do you think is going to happen when someone forgets their key and can't send an x-ray to the remote radiology center? I have my guesses.
If user will have to enter 16 charter password each time after HW key (like Yubikey) will be connected to a computer to unlock it, then users will leave it always inserted. Or password will be saved in a text file. If HW key will just work once inserted (or will require 4 digit pin) most users will comply. It is already 2nd factor in addition to some other password, it doesn't necessary need a strong password to use it.
Depending on the precise scenario, that may or may not represent an improvement. If the key is used as a second factor to authenticate to the network, then an infected Excel document will trivially ignore the involvement of a Yubikey as it uses the logged-in user's Kerberos ticket to spread.
You're completely right, though. Even this would definitely cut down on phishing attacks that send users to fake websites pretending to be internal systems.
I'm not suggesting we should declare anything impossible. Far from it! I'm merely trying to suggest that we should appreciate that not all things as easily fixed as they may seem at first blush.
As all of us in software know, complexity can lurk in unexpected places.
I'm a security specialist and I honestly wouldn't know where to begin.