I totally understand it's not an easy thing to do, but many things in life that are worth doing aren't easy.
My home (Arizona) has a similar reputation now with the SB1070 flap, and make no mistake, it has hurt recruiting.
But face it, there are significant "network-like-effects" in places like Silicon Valley. The talent (and money) tends to congregate. Not saying there's no talent, or strictly lesser talent elsewhere, but you can't argue with reality.
As someone who just moved from a nowhere town to Austin, and managed to bring three people with me, this is very true.
My parents were enticed by a job offer to move out of the SF bay area. The offer was better money, and all moving expenses paid, but they ultimately declined. Their reasoning was that for their type of jobs, the SF bay area was the best place to be unemployed. So they stayed.