Permits cost tens of thousands of dollars, and months of processing in my area. They’re happy to sign off at the end after they’ve extracted the maximum they can from you.
OTOH, insisting on permits potentially saved my ass at least in one case - I had a contractor remodel my bathroom say that "they do everything up to code". Their work did not pass inspections on the first try (had to fix things up both for plumbing and electrical).
The intent in many jurisdictions is to tarpit change as much as possible and it shows.
It isn't the permits themselves which are usually the problem.
I was paying for a house to be built by a contractor who, it turns out, screwed up the permitting process and didn't do a hydrological survey of the erosion pattern on the land. The town yanked his permit to build.
I was grumpy, but when I drove by I could see clearly what the problem was: he hadn't planned for drainage at all, and the new construction and alteration of the terrain had already caused rainwater to start pooling at the base of my future neighbor's foundation. If he'd been allowed to continue unhindered, he would basically have guaranteed my neighbor's property would have been destroyed in 5 years.
It really isn't a box for $300k. You can buy a mobile home for what $60k?
Permit process isn't broken.....this is a view often held by incompetent contractors. I have performed various jobs like building retaining walls, bathrooms, electrical, decks and never had an issue as long as my work was up to code.
Permits can protect a homeowner, when the contractor bails without finish the work.
But back to why a home can be $300k, it is a lot of work and materials. From an article, "According to NAHB, the average material cost to build a house is $296,652, with the average square footage of a house being 2,594. That means your cost per square foot is $114.36 ($296,652 / 2,594)."
You are underestimating how much it costs to get lumber, copper(can use pex), brick, concrete etc.
You sound like someone saying, I can have a wordpress site for $1000, but why does it cost $500k to build an ecommerce platform.