> because someone decided to come up with a way (race) to divide people into groups where one group is bound to end up smaller

Well, yes, and this was codified into US law until the last bits were finally suppressed in the 1960s, but that doesn't mean the underlying racism has been eliminated as well.