Half of California's tax revenue comes from the top 1% [1]. Medicare retirees are almost guaranteed to not be part of that 1%. You're talking huge tax revenue losses here if the 1% do decide to leave.

[1] https://lao.ca.gov/LAOEconTax/Article/Detail/7