As an example, I'm a white male without college experience, raised in a single parent, lower income household. Most people hiring for "diversity" would overlook me just because I'm a white male, but would welcome my wife's vietnamese friend, even though she comes from a wealthy background and went to a prominent school that makes her much less diverse culturally than their existing employees.
I very much want to weed out bad experiences from turning otherwise interested contributors away, and strongly support that. But when it crosses a line to superficial tokenism, we all lose, including those who are underrepresented.
Assuming that someone's demographic means that they think differently about a topic to another demographic strikes me as very uncomfortable. Basing an argument for diversity on the assumption that e.g a black female programmer is unable to think about a problem in the same way as a white male programmer doesn't feel like a step forward.
I think you may be being too absolutist here? This would only be true if literally every demographic thought identically about every aspect of a project or product and brought the same point of view. Then everyone would be interchangeable beyond pure technical competence. However I don't think that's always true, and when it's not that means that diversity can have some value purely in and of itself since it will bring a better reflection of global usage goals and UX considerations.
I want to emphasize that this doesn't mean it's a more important value then anything else, which might be where some of the reflexive opposition comes from. It shouldn't be a zero-sum game, where gaining diversity means necessarily losing on other important values. But neither is diversity never of any inherent positive value, it can be, and that in turn is worth some positive effort to pursue isn't it? Not merely correcting biases, but actively seeking a wider array of PoVs from intelligent people could help avoid mental boxes and unpleasant surprises when something goes out into the general world.
So maybe a better version of what you said would be "making diversity the only goal" or even "the goal above all else" or "diversity a goal but never taking into account whether any cost/benefit tradeoffs to existing culture makes for a net win" or something along those lines?