Maybe not your grandparents, but maybe their parents. Harry S. Truman could not actually afford a house until he was in national politics ( perhaps even POTUS; a quick Google didn't clear that up ). 100 to 40 years ago, people would only pay a year's pay, two years for a house.

A lot of the cost of housing is due to the 30 year fixed mortgage. When you subsidize something, it costs more and you get more of it.