You don't know that.
When has it ever been tried?
We didn't explore societies where transactions are voluntarily (eg. no taxes) and we don't persecute victimless crimes (eg. drugs)
I argue my interests are aligned with the majority of non political people.
That said I'm not a proponent of crypto, I think private banks without government regulations are a more efficient alternative than a blockchain.
Chaum's original digicash worked something like that. It could only be transferred anonymously once. It could only be converted to and from regular cash at regulated entities like banks. So you'd buy some coins at the bank. You could then use the one anonymous transfer to send them to some youtuber. Neither the youtuber nor the government would have any way to learn who sent these coins. But, the youtuber would have no way to send them to a third random person. The only thing he or she could do with them would be bring them to a bank and convert them back to normal currency--along with doing any required tax reporting that is required when you bring cash to a bank. All this was mathematically enforced by the crypto protocol (blinded RSA signatures). It wasn't that great for libertarian madness like Bitcoin supposedly was, but we've since seen how that turned out.
Maybe it's time to revisit Digicash, especially since the patents have expired.
Nothing in the Digicash blinded RSA signature scheme would prevent the tokens from being traded person-to-person any number of times before being presented back to the issuer. It requires some trust in the previous token holders, much like exchanging BTC by trading private keys in place of on-chain transactions, but it's perfectly doable given some other means of discouraging defection. And of course the blinding system itself ensures that the tokens being deposited can't be linked to any particular withdrawals.
Edit: In the case of off-chain exchange of Bitcoin wallets it would reveal the private key (naturally, since that's what you're exchanging) but for that reason you would only trade a wallet with a unique private key not used anywhere else. The only thing that key is useful for is spending the funds in that wallet. It's not linked to your identity. This is exactly like the "physical bitcoin" model (BitBills, Cascascius coins, etc.) except there is no tamper-evident hologram to prove that the private key hasn't been accessed—you're taking that on trust until you empty the wallet or hand it off to someone else.
... that's you're right but 'we' don't care if you don't care.
We know that without decent regulation, tax evasion on a massive scale, money laundering, criminal activities, insider trading, bribery, extortion, illegal political influence etc. etc. will seriously degrade civil society.
So 'we' put some regulations in there for those reasons.
And without attributing financial transactions to specific legal entities, there's no way to facilitate commercial or litigation otherwise, thereby destroying trust in even regular business transactions, which makes commerce impossible.
'The Means of Payment' is the mechanism by which the transaction is effective, so we'd rather have that in place or not.
For small sums, under $500, it's unlikely we need to worry, but beyond that it gets tricky.