What other artificial distortions are you thinking of?
A full discourse on HN isn't really the right format and forum to discuss land and socioeconomic policy, unless in the context of programmable shelter construction or similar. So I'll offer a tip of the iceberg thumbnail sketch. Land as an asset is given special privileges that other assets do not enjoy, and tightly coupled with that is debt (leverage for land) and capital are treated with special privileges that also are rarely found in other asset classes. These privileges are often enrobed and justified "for encouraging economic growth and development", but while that was true at inception, reflexivity has rendered those rationales moot today.
Personally, I've studied enough to realize I don't have the power to materially change any of this, and instead I focus upon what I can change in my individual sphere of influence. For my personal setting, that means focusing upon modular, incremental improvements in opex and deliberately eschewing conventional norms of how to allocate capex. A concrete example is eventually spending more on foamed glass insulation over time as the original shell itself, with the realization that easily-maintainable 70+ R-value insulation over >100 years between replacements will pay for itself on opex savings alone. A tangentially-related concept is reducing the economic churn in your and your family's lives as a means towards the cost avoidance of the associated economic rents extracted from the churn.
Housing affordability is complex issue. While there are local factors like zoning as you mentioned, artificial distortions in the open market for land in the US (and much of the rest of the world) contribute a plurality of the effects causing the stratospheric cost of land wherever well-paying jobs can be found.