Let's not forget that most white men on Earth were always in the "oppressed" group. In Europe, you were either in the aristocracy, in some semi-privileged class like clergy, or you were working super hard for a living, and sometimes (fairly regularly in some countries, e.g. France) died due to starvation. In America, it was similar - yes, there were some slave owners, but majority of white men were living and working in abhorrent, dickensians conditions in the slums of New York etc. or living a tough, poor and primitve live rural life, while being opressed by nature itself.