This can be partly explained by the fact that much racism in America actually uses cultural markers in addition to just skin color. For example, many people will judge a person as stupid if they seem "thuggish" regardless of skin color. Some don't consider that racist, because it's not about skin color. But it is a cultural marker that correlates with race, so in that sense it is at least partly racism.

Someone from Africa would only have some of the cultural markers we look for when making racial distinctions in America, so in this way they would be partially spared. Sometimes all it takes is some indication that this black person is "not like all the other black people" for someone to be open to the idea they are a capable and safe human being.

Our prejudices are not simple. We are not stupid, we are very good at seeing patterns. And evidence for intra-racial distinctions is not evidence against interracial distinctions.