Main issue in Palo Alto (and to a large extent the rest of the bay area) is that there's massive population growth in the bay area, but very little housing is allowed to be built. This creates artificial scarcity and the people that bought houses 30yrs ago accrue massive wealth because of it at the expense of new people (and the new people that manage to buy at exorbitant rates also end up paying the majority share of taxes too).
This is the natural incentive made worse by things like locked property taxes and rent control. You really need legislation that allows the market to build (easier said than done).
Remember that even when you aren't hampered by lack of space and/or zoning, most build starts are still 'investments'. I knew a whole lot of folks who did homebuilding in the first decade of the century. Housing starts were all over the place, partially because the investors were sure that with prices rising, it was a sure thing, right?
But then the market glut happened. Housing starts stopped. All of those folks left homebuilding and few if any ever went back to it.
I think nowadays Investors are back to preferring smaller buildouts, potentially in more phases. The supply/labor issues from covid haven't helped things lately though.
The rules created by existing residents where designed to push up rent and home prices, and they have succeeded.
The biggest expense is typically housing. And if you bought the house, it means the biggest expense is... building equity into an asset you can sell afterward.
Keep in mind the engineer who purchased a 2 million house in Palo Alto can sell it for two million, and then move to Ohio to a much cheaper house purchased in cash. The reverse isn't true.