You're wrong, there was a large financial incentive to abandon neighborhoods once non-whites started living there. The racism caused housing sell-offs, amplified by the crashing home prices. People quickly learned that their home value would plummet when the neighborhood became integrated, so they rushed to be the first ones out; while values where still high. This, of course, exacerbated the problem. Racism is at least a good 70 to 90 percent of the problem.