I think the person you're responding to is assuming that we are already at a decently fair equilibrium, and that diversity and other concepts and goals are disrupting that equilibrium. I think the biggest failure of communication around these issues is that a lot of people arguing against these programs are coming at it from the just world hypothesis.

https://en.wikipedia.org/wiki/Just-world_hypothesis