https://gist.github.com/hughdbrown/94016d0f477b855a893514131...
Sadly, Salt Lake City is dropped from the set because of the missing foreign-born data.
If we were to dramatically over-simplify by assuming a city's economic mobility is a simple sum of independent Gaussian variables, is the following correct?
(1) Pearson and Spearman rank correlation are the same.
(2) Percentage foreign born is more important than all other variables combined (since they would contribute 39% of the total variance)
I don't know what this means -- like you'd get the same value if you calculated the correlation by Pearson correlation and Spearman rank correlation?
Or are you saying there is something called the Pearson rank correlation and asking how it relates to Spearman rank correlation? I have no knowledge of a Pearson rank correlation.
I used Spearman rank correlation because it is applicable to this situation: it "assesses how well the relationship between two variables can be described using a monotonic function." [1] Pearson's correlation would be appropriate if I was looking for a linear relationship between two numerical values, which I don't have here.
> Percentage foreign born is more important than all other variables combined (since they would contribute 39% of the total variance)
Hmmm. I think that it is the r-squared that is the proportion of variance in Y that is accounted for by X. Accordingly, the Spearman correlation of 0.61105258893120507 means that 37.3% of the variance in Y is accounted for by X, so really there is 62.7% of the variance still to be accounted for by other variables. There could well be another variable that explains the relationship better. [2]
[1] https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_...
[2] I am assuming here that the relationship of correlation to variance for Pearson correlation works the same as for Spearman. I think they should, since Spearman rank correlation is just Pearson correlation applied to pairs of ranks.
I mean: under the stated assumption that the data is given by a sum of independent Gaussian variables, would the Pearson correlation of the Gaussian data be equal to the Spearman rank correlation of the corresponding ranked data.
One can check from the definition that the answer is "yes".
> Hmmm. I think that it is the r-squared that is the proportion of variance in Y that is accounted for by X.
Yes, you're right, I needed to square this number. So, assuming Gaussian independent variables, 37% of the variance in mobility is explained by percentage foreign born. It would be pretty surprising if there were even an larger effect independent than this, so I'll self-grade my conjecture as "plausible".
https://www.brookings.edu/opinions/the-economic-case-for-soc...
Growth and social mobility are the things Salt Lake and San Jose have in common.
The economic engine of San Jose relies on recruiting immigrants to drive a young, skilled labor force. In Salt Lake they manufacture their own youthful workforce by having the highest fertility rate in the land.
Think of the baby boom -- the last time both growth and mobility were high in the US.
The mechanism is irrelevant, but I'll point out that both Salt Lake and San Jose are running pyramid schemes. Eventually you run out of places to put people and then your population ages and then you have a problem. See Japan. Also, the bill is coming due for San Jose sooner and for Salt Lake later but that's another story.
Oh, and I completely reject that Mormon institutions (while perfectly admirable) and lack of ethnic diversity (questionable at best) have anything to do with Utah's recent economic pop.
When you only take people who meet the H1-B requirements, of course it is the case that the percentage of foreign-born people is strongly correlated with upward mobility; all of the foreigners move upwards.
Let's check: Out of the 50 largest cities, here are where those high-mobility cities rank by percentage foreign-born.
and Charlotte is #25 with 13.7% foreign born.https://en.wikipedia.org/wiki/List_of_United_States_cities_b...
So I'd say this conjecture looks pretty good for explaining cities that have the very highest mobility.
(I could not get foreign-born data on Salt Lake City because it's population is less than 200k. I don't understand why your source would list it as part of the "50 Biggest Cities in the US," since the US' 50th city is Arlington, TX with ~380k.)