I see how that could happen, and probably has. But, for example, I can also see how the top x black applicants to Harvard would be capable of doing very well at Harvard even if those applicants are not in the top 4.6% of SAT scorers who apply to Harvard. (where x is maybe 14% of entering class size to be proportional to US population, and 4.6% is I think the Harvard acceptance rate.)

Anyhow, it's a tough problem and I don't know the right answer and don't envy anyone having to set admission policies.