That was only true in some states during Jim Crow. Others had different definitions of "white". In fact it was possible for some "black" people to become "white" just by crossing a state boundary.

Racism is stupid.