It's not the race, it's the economic and class changes that happen when whites move into a mostly black or hispanic neighborhood. And concerning US cities, it's pretty much never an instance of middle/lower class whites moving into a middle/lower class black neighborhood.