This is a really good illustration of why it matters to be aware of what your sample sets are, especially in the case of using machine learning / automation in regards to things that have a very high likelihood of impacting human rights.

To be honest, I don't think our understanding of these systems is mature enough for us to be just throwing them out into our societal systems right now. There needs to be a lot more testing because the possibility of unethical results is pretty damn high.

That said, I still think that the concept of structural racism is a bad way to look at this problem, as it's simply one form of a common error when looking at sample sets.