I would posit - admittedly without data - that outside of major metros where land is the clear dominant cost, most costs for housing increase is labor.
There are houses and empty lots available within 15min of downtown Cleveland that are selling for <50k. If you go up to $100k, there are dozens of houses. They’re all empty lots or houses that need renovations. If you want a new construction in the same neighborhood, it’s $300k. The cost difference is labor or materials.
Obviously Cleveland is a cherry-picked example but I used to live there so I picked my old neighborhood. San Francisco, Boston, Seattle, etc are obviously mostly driven by land costs due to demand and very high incomes.
All that said, some parts are gentrifying. Though mostly where developers can get large enough plots.
The point is that most of the cost of these homes are labor - even when land is nearly free (<50k in a city), you can’t get a “cheap” home that’s made to modern standards. If the land is free, there is no demand and high supply, but the house is expensive- it’s labor or materials driving the cost.