> it's completely possible that a major cause is that the median in that arbitrarily defined racial category is below the median in another arbitrarily defined racial category.

Sure, possible, in the sense of theoretically possible. But very unlikely as socioeconomic factors potentially dwarf all of the aggregate group differences.