My threat model does not include "Nation state adversary breaks into my home and... reads the contents of the book". If I annoy the Russians enough, presumably they would just try to outright murder me.
In contrast, "Person I annoyed online plans elaborate Internet revenge" is definitely a potential threat I want to cope with, as are "Scam email claiming to be from company I have account with", "Facebook lose everybody's passwords", and so on.
If you meant somehow the building burns down and destroys all my possessions but leaves me miraculously alive, I lose access to a bunch of stuff. Nothing to be done about it.
But one of the Security Keys lives in my jeans pocket, so if I survive the fire in the regular way, by fleeing a burning building, I still have that Security Key. If I am wearing trousers. Not so many people actually flee naked, if the fire is going to kill you in the time it takes to pull jeans on you're probably just not making it out at all.
On a flash drive/SD card, or even printed out, and then stashed somewhere safe/secure (i.e. not in an unlocked drawer next to your desk)