I believe it was more complicated than that, with some moving around since the 1930’s at least. I think the 1960’s civil rights movement just made it painfully obvious.

Having said that, Americans seem to be extremely pragmatic when it comes to voting, not ruling out someone for being a terrible human being if their policy proposals make sense to the voter. I wish Americans would be more ideological and principled in that situation.