It's not a single axis of (dis-)advantage. A wealthy black person in America has economic advantages over a poor white student, yes. But the black student will have disadvantages based on their skin color that a white student wouldn't have. There are a lot of people advocating for entirely replacing race-based affirmative action with economic/wealth-based weighting but they're not mutually exclusive.

I would argue that wealth is significantly more important than color of skin.

Yes race matters. But your social class and financial means matter far more in most casss.