That’s an assumption not directly based on numbers. Yes, the net in migration to California is largely from non-domestic people. That could be due to exactly the reason you provided, or it could also be the result of less movement freedom tied to visa sponsorship, or a variety of other factors. Many of my tech coworkers in Texas are non-citizens under visa sponsorship who, given the choice, have stated they prefer the Midwest due to a variety of economic conditions (not just cost of living).

Speaking of unsupported anecdotes, it makes sense that the people you meet in the middle of the country express a preference for being there. That's pure selection bias. In terms of revealed preference, the states that are getting more than their per-capita share of California out-migrants are Nevada, Arizona, Oregon, and Washington.