I think much of the problem is that these toxic effects aren't being created by overt bigotry, but rather by cognitive biases. The people rescinding reservations are doing so out of loss aversion, which is a very powerful human impulse. We're also powerfully geared to think differently about outgroups than we do about ingroups, which makes it easy for us to believe that there's a higher degree of risk in allowing a black family to stay in our house than a white one.

Social stigma might blunt these effects but it's unlikely to eliminate them any time soon. I bet a lot of the people who (effectively) refuse to rent to black people don't think they're the slightest bit biased against black people. They're not keeping track of their individual decisions about rental applicants, and they can easily let their brains fill in the blanks to tell them there were all sorts of negative signals they picked up on when declining reservations.