It's quite possible that the 'California wealthy' are a minority of transplants (say 20-40%), but that could still move real estate markets significantly. Looking at [1], it seems like the Portland housing market does about 20-40k sales per year. And it sounds like Portland has been growing by about 30-40k people per year (that is total, not just Californians). So if 25% of those folks are jumping into the real estate market, it can probably have a big impact on the supply/demand. Housing is pretty slow to respond to supply/demand imbalances so even a small imbalance can swing the market.

[1] https://www.trulia.com/real_estate/Portland-Oregon/market-tr...