Most developed countries took a significant hit to fertility rates around 1970-1980 and stayed below replacement rate since then, fluctuating or dropping further down but never recovering well above replacement rate. I can think of a few things happening during those years, but "housing" wouldn't be a global one.