Perhaps lead levels contribute only a small amount. But then also, micro-aggressions have a tiny affect. Testing bias might also be very small. Maybe less income also leads to being less likely to get glasses when you need them. Perhaps food deserts in minority communities leads to poor nutrition and poorer outcomes, but only a negligible amount. Then there are poorer, urban neighborhoods that might be loud at night causing children to not get quite enough sleep.

And sure, you could blame a lot of this just on poverty in general, but then of course, you have to acknowledge our racist history that lead to so much less wealth in minority families.

It goes on and on and pretty soon you have death by a thousand cuts and a phenomenon that is far too complex to explain in anything less than an entire encyclopedia so you sum it up with "systemic racism".

Note that tiny fractional differences in the median outcome gap will turn into big differences at the left- and right-wards tails of the distribution. In practice, those are the differences that everyone will ask about. Why are so many X's (relative to Y's) living in socially marginalized settings, resorting to crime, ending up in the justice system? Why are there so very few X's in very high-skill, high-effort occupations like computer programming and other hard-STEM fields? The X's and Y's will vary by time and place, of course. And societal oppression of various sorts is always playing a role in these dynamics, if only as an amplifying feedback. But either way, these are questions about uncommon, tail outcomes not average or median ones!

That's true, but the gap is already huge at the median. As a rough approximation, the distribution of outcomes has same variance, but mean is shifted left by 1 sigma.