A number in your head is portable. So with the exception of the 'border crossing' use case, I agree they are functionally the same thing.
I suspect a lot of more of the 0.1% have gold hidden than many would expect. I've met two eccentric 100 millionaires that advocated for hiding gold as a wealth preservation mechanism. I think this is becoming the primary non-trading use case for Bitcoin - met both of those individuals through conversations about crypto.
Bitcoin is subject to federal policy as much as anything is: as much as secret Swiss bank accounts.
You imply evasion works. But that certainly doesn't mean it is federally legal, legal with the IRS, or legal for whatever other reasons. You have the risk of serious consequences if you are caught breaking the laws that apply to you.
Your argument implies that you can break the law without getting caught, but we already have surprising crypto counterexamples against that.
Also, the market is slowly being split into legitimate trading (via centralised platforms with KYC rules) and illegitimate trading (for stolen bitcoins, or bitcoins tainted by tumbling with illegal bitcoins). As that plays out, it seems likely to become more difficult to actually trade your crypto without some laundering costs and risks.
Keeping a number in your head or hiding some gold in your secret location are the same thing.