House prices went down in the 80s (although were still quite unaffordable). Additionally, those who bought in the 80s got the ride the gravy train of lowering rates over the decades which massively boosted prices and allowed them to refinance.