Existing housing is expensive because a substantial portion of the stock isn't owned by the residents. Housing is used to extract rents, by wealthy individuals or businesses, and rent-seeking drives up prices.

New housing is expensive because of safety regulations, environmental regulations, and zoning restrictions. Those categories are listed in increasing order of negotiability: some zoning makes obvious sense (e.g. separating industrial areas from housing), but housing density mandates seem less defensible to me; meanwhile, I don't particularly feel like relaxing safety unless something can be demonstrated as clearly irrelevant due to age, or requiring gross overbuilding. I invite someone more fluent in the subject matter to enlighten us with the relative impact of each factor.