No it won't. If society builds more dwellings society is richer. Wealth is the actual physical production of goods and services.

We wouldn't get richer if we blew up car factories to protect the market value of existing cars. The market value of existing cars WOULD skyrocket if we destroyed all new car production, but that would be society getting poorer, not richer, as we would have fewer cars.