> half of what's remaining has to be saved up for retirement in a place that deeply discriminates against age, you don't have much left to save up for buying a $2M house.

Buying a $2M house is saving for retirement. Owning your own home in retirement is a massive financial win. And if you own a home in an expensive market, you can sell it, buy one in a more affordable market, and then the difference is unlocked retirement savings.