I know plenty of diverse liberals (the conservatives already fled) who are moving outside CA due to tax reasons and housing reasons. These are not rich people but a range of professionals in a handful of salary brackets.

So it's not just Elon. It's those who have the opportunity to do so where it does not impact their lifestyle significantly.