As Sean mentioned, not only have we increased in total population we have also changed where we are living. Lots of small towns have seen their populations decrease, with some completely disappearing. Large neighborhoods of cities like Detroit and elsewhere essentially emptied. We need more housing and we need to shift housing resources to where the demand is.

There are loads of cheap houses in the USA. They're just in places where most people don't want to live.

Here's a cheap house:

https://www.zillow.com/homedetails/420-Tyler-St-Gary-IN-4640...

The commute to Southern California is pretty killer though.