There is no such thing as "invading" your own country. The level at which we can make such claims, and implement such controls, is the national border.

The equal protection clause of the Constitution establishes freedom of movement for all Americans. Native and longtime San Franciscans have no more or less right to be there than the Iowans and Wisconsinites flooding in.

If you want to change that, then secede from the United States.