I don't think the two hands can properly "validate" algorithms without communicating. The algorithm designer can maximize AUC, but what if one <group>'s class is 95% label A; the designer always predicts A for <group>. How bad is ALWAYS missing 5% for label B? If you can put a price on it, then the developer can build it into the algorithm. But if the price is difficult to accurately estimate, or non-monetary qualities are desirable, it may be hard to build them into the classifier ahead of time. On the other hand if the cost of perfect <hard to quantify criterion> reduces AUC significantly, algorithm designers need to communicate that...
Humans produce the data that the ML algorithms train on, so without whitening the data somehow (and there's another contentious issue) we should expect the result to be biased as well.