House prices have indeed gone up, but there is no guarantee that they will continue to climb. There is a limit to how high they can climb relative to income. You also picked one of the hottest real estates markets as a point of reference. The returns in many other places are much lower. As a comparison, the S&P 500 with dividends reinvested over that same period (january 1996 to january 2017) has returned 440%.
When buying a house, timing is crucial for the return. Renting and monthly investing in the stock market has the advantage that your investment is spread out in time.
For most people, the biggest advantage of buying a house with a mortgage is that it imposes a discipline: you have to pay off the monthly instalment. If you rent you might be tempted to spend the rest of your income instead of investing it.
Both have advantages and drawbacks. I just don't think it is the no-brainer you make it out to be.
That doesn't necessarily make it one of the best ways overall, but it is a very good way if you're starting with little because it's basically your only option. Trying to scale that up (into real estate beyond your personal residence, rental properties, etc) is pretty close to buying yourself another job. If you work at that job diligently, you will do better in rental property than in your own residence, but it's work.
For those with some capital (say 1+ million USD) and a full-time job, real estate as a second job is not a particularly good fit IMO. The un-levered returns you can get in the stock market are better and the time commitment is largely optional and can be at a time of your choosing.
The long-run real returns to real estate are essentially zero. TINSTAAFL.
The only reason it's "profitable" for the average person is because the financial system allows for easy leverage (home mortgages). Take away that subsidization and real estate is just another asset class with no advantages.
>Homes in many regions of California have increased 3-4x since 1996
And many random stocks have increased 3-4x since 1996. So what?
Leverage is only an advantage if your return is higher than your cost of borrowing.
Which it is for housing, and many other investments. Only the bank won't lend you $500,000, backed by the government, to buy stocks.
My qualm with the OP was the implication that real-estate has outsized returns. Unless you're taking advantage of a home mortgage, there's nothing significant about real-estate returns. Of course most of us here are of the generation that saw nothing but rising house price (except for that one time where they didn't, but that's an anomaly, right?)
Avoid foreign real estate though unless you are very careful