It would cost more, but materials are the largest cost. A significant part of the labor are more skilled jobs that illegal immigrants can't do - plumbing, electrical, etc.

And if people aren't building because it makes more sense to buy a "used" house material prices would also drop.

Materials aren't the largest cost around here (California) There has been a massive worker shortage since 2008, so even incompetent people can charge way more than material costs for most installation jobs. (And then you pay a second person even more.)

Permitting, engineering, etc are also expensive.

Finally, construction costs are difficult to estimate up front, so it is difficult to be a rational actor when deciding to build or not.

After all, if you hit a 33% cost overrun half way through building, your best option is to pay the 33%. At that point the house costs 83% of the expected price of starting over. If you stop now, then you are out 50% of the initial budget.

If you think facing such a tradeoff is rare, look around for lots that are for sale with pre-approved plans or partially constructed houses. They're usually pretty easy to find.

I heard a quote from roughly a hundred years ago, from a US immigrant. Paraphrased from memory:

I came to the US and found that, not only were the streets not paved with gold, they weren't paved at all. Furthermore, I was the one that was expected to pave them!

I don't think much has changed on that front.