Yeah, in Cheek v. United States it was found that (in an exception to the normal rule) ignorance of the law is a valid defense in criminal tax cases due to the complexity of tax law.

https://en.wikipedia.org/wiki/Cheek_v._United_States

https://www.irs.gov/pub/irs-wd/0947055.pdf

Yeah, my experience with making mistakes on my taxes was that they sent me a bill for the difference, I paid the bill, and that was it. There wasn't even a fine beyond that I had to pay interest on the money I owed.

Sometimes the IRS is wrong, too, and you just mail them proof. Like if you forget to report your cost basis for a stock sale, they'll send you a letter and say "We're assuming the cost basis is zero, so here's what you owe!" and you just respond back with a letter that says "This is the cost basis, and here's a copy of my brokerage statement." and that's that. I've found dealing with the IRS actually quite straightforward and non-scary. As long as you have your paperwork in order. If you're careless about your records, then you're going to have a bad time.

Exactly this. Is what I meant by them working with you. They aren't playing some devilish game of trying to trick you into paying more. They tell you why they say you owe something and what it would take to get them to change that. And are fine if you need time to find it.

If you are actively trying to avoid the calls, I'm sure that is probably not a good idea. But, that is basically in the "actively trying to commit fraud" territory, at that point. No?