What did a person making $1m/year do wrong they are penalized unfairly against another person that doesn't? I have a feeling, much like now, all this accomplishes in reality is the rich hire good enough accounts to hide their largest sums from the IRS. That leaves the middle class and poor once again bearing everything... and we're back to where we started. Carving out 0 exceptions and sticking to it ensures that that literally nobody can say they are being treated unfairly.
My previous point wasn't anything about "should", by the way. I was merely saying that, in the current political climate, not hitting the rich harder was something that would make it harder for a VAT-only scheme to become reality.
Note that a lot of voters are convinced that they will make $1 million/year some day. Perhaps not today, but some day, and they seem happy to pay more today to ensure that they are not "penalized unfairly" when it's their turn.
This is not necessarily a rational approach, since very few of them will ever earn that much money. But if your goal is to get votes, you can go a very, very long way with "penalized unfairly". Note how many oppose an inheritance tax that they'll never pay, simply by the expedient of dubbing it a "death tax".
The "penalized unfairly" argument wouldn't wash with most economists, but there's not all that many of them. And nobody really listens to them anyway.
But that won't get you where you say you want to go. Who's going to collect that VAT, and make sure that everybody pays it? The IRS. Your approach would get rid of the tax code, but not the IRS. (In getting rid of the tax code, it would let you get rid of the computer code in question, though.)
The other problem is, you can't implement a "soak the rich" approach with a VAT. And currently, there's a lot of angst around wealth inequality, and most plans to do something about it involve taxing the rich more than everyone else. So your idea is fighting an enormous political headwind.
You could propose something like VAT + income tax for those who make over $1 million/year or something...