We don't need to speculate. There is strong support in the historical record that many cities adopted single family zones for explicitly racist reasons.