Some combination of temporarily embarrassed millionaires and tribalism. They may think of themselves as future rich people who can benefit from cheaper labor, and they may be step above the lowest on the socioeconomic ladder, and so want to maintain that hierarchy. As President Lyndon Johnson said:

"If you can convince the lowest white man he's better than the best colored man, he won't notice you're picking his pocket. Hell, give him somebody to look down on, and he'll empty his pockets for you."