> You realize that you still are not selecting the most skilled candidate if you filter based on diversity at any level.

This depends on the size of your sample and the positions to fill. It is easier to find one good candidate from 100 individuals than 100 from 1000.