Explainable models do not preclude the systemic problems you highlight. Plenty of systems before the advent of non-explanatory ML models had those defects. One option is to define test and validation sets and encourage 3P validation, somewhat like how accreditation works in other contexts.