Did the evaluation of algorithm's superior performance included both false positives and false negatives?

Personally, I don't like the concept of deploying algorithms for which we don't understand how they work, in scenarios of high health or social impact. It's cute to deploy DNNs to classify cat pictures, as when - not if - it spectacularly fails, nothing of value is lost. But if you can't point at the verified model the algorithm follows, or at least trace the algorithm after the fact, understanding and assessing every step it took, then that algorithm has no place in making decisions about people's lives.