Buying a house or renting housing leaves a paper trail. So does renting or buying a car from most places that sell or rent cars. So does paying utility bills. So does paying for prescription drugs, medical services, car insurance, life insurance... So does paying for higher education.
I guess if you had some cash-only tax evading side business you could use cash to buy things like food, clothing, gasoline, and miscellaneous household goods. But you're not going to be able to dodge the taxes on middle class life's major expenses -- at least not without tripping the same mechanisms that catch drug dealers buying cars etc. with cash. I suppose you could live like an off-the-grid survivalist who never interacts with ordinary businesses. That looks like a lot more work than living an ordinary life where you work at jobs that don't involve trusting other criminals, buy things and pay taxes like usual.
You're also assuming that the government would sit still in the face of a serious erosion of the tax base, and not crack down on it. The US is bad enough with its arbitary seizures of cash, let's not destabilise the situation further.
Sure, you still need to buy things in person, which can be done with cash. But drug dealers seem to be able to do a pretty good job of evading taxes.
Imagine if everything that you bought in person was done with cash, or a bitcoin payment. That money would be untaxable.
If you buy a house, for sure, the government can track that, and you need to make sure the money used to buy that is "clean", but a surprising large percentage of a person expenses could be done in a safe way.