Anecdata: I have a very complicated return due to self employment while overseas, and it takes 3 solid days.
Of course, most of the compliance burden is on tax professionals working for business.
I think we're probably losing 3 to 6 new software features every year due to the administrative burden on you alone. We all lose out when the labor of any of us is destroyed needlessly.
I'm self-employed, but in a relatively simple situation as such things go (single-member LLC), and, aside from my mortgage, no non-trivial assets or investments. It still takes me a solid day and a half to put together my return, and a few additional hours over the next 1-2 days sorting out fine points/details.
I cannot imagine how long it takes someone with property, 401ks, etc.