I assume the cost of bureaucracy is what we collectively pay for in taxes. If what I pay in taxes for these public services is cheaper than purchasing these goods and services through for-profit vendors, I think I can safely say I do not get a similar cost in bureaucracy.
Additionally, even if these things were slightly more expensive than the "free market" I would be willing to pay more and help share the burden of ensuring everyone had universal access.
https://en.m.wikipedia.org/wiki/United_States_Postal_Service
Private schools are on par with public schools ($10k/yr) for the same kind of school, with large class sizes. Expensive private schools are far better "product" than public schools with small classes and better extracurriculars and amenities.
Your property taxes are subsidized by taxes paid by non-school-families.
Are you saying if the USPS didn't exist FedEx and UPS would be cheaper?