Personally I think hardware wallet random numbers are safe but anyone technical who's worried about that could generate their own from von Neumann coin flips, and turn that into a 24-word seed for setting up a hardware wallet.
https://en.wikipedia.org/wiki/Fair_coin#Fair_results_from_a_...
I wouldn't say this is more difficult than securing a large amount of gold. If someone gets into your safe deposit they can just take your gold. With a hardware wallet you get three guesses of a 4 to 8 digit PIN and then it wipes itself. And you can't store gold redundantly to protect from loss.
Anywhere in the world your securing Bitcoin against a billion or more people, plus bots. A proven attack vector could wipe out thousands of people in afew minutes.
> wipes itself
Now that seems like a really shitty security feature. Whoever finds it can just wipe out your funds?
Now, given that I have recovery keys stored in a safe place anyway, the ledger itself is somewhat disposable. Its still a great way to actually access my money, but if something happens to it I can just get a new ledger. So, yes, I want it to wipe itself if the wrong PIN is entered a few times. That feature means if I lose the ledger I can relax knowing my money is still safe even if somebody finds it.
That is good for someone who have several secure redundant copies.
It's hard to steal a truck of gold, especially considering the mens with guns which are typically around. It's hard to mess up physical security of a truck (not even to mention a vault), and even if you lose it, the thief still has a big and heavy problem on his hands. Real world police is good at these things. And unlike bitcoin, gold can't just teleport to Russia (using a generic non-cooperative country here)
Stealing bitcoin requires just one tiny mistake in a miriad of places (starting with random number generation, ...), and then you are done with no other recourse.