The ethical concern would be, "don't build a racist AI system that assumes black people don't pay back loans". This could be mitigated by removing race and clear proxies such as zip code from training data, but it doesn't necessarily break your cycle, because the poverty and social problems are real and exist regardless of whether the AI considers race.
The anti-racist approach, on the other hand, would specifically require incorporating race so you can perform affirmative action. Maybe that's good for society, maybe there's a debate about how much and in what way you do it, who pays for it, etc. But it's left "ethics" far behind and become "politics" at that point IMO.
It isn't this simple. Not all these proxies are clear. Also what are we even left with once all these proxies are removed? The zip code is hugely important when trying to establish the value of a home. Does removing all data that is potentially racially biased leave us with a model that is nearly worthless?
>But it's left "ethics" far behind and become "politics" at that point IMO.
It depends entirely on what branch of moral philosophy to which you subscribe. For a lot of people their personal politics is just applied ethics meaning not acting would be ethically wrong in their opinion.
I mean it is never truly the "AI's fault". It is the fault of the systems that resulted in the biased data and the people who ignored that bias while still delegating the decision making to that AI allowing it to be a tool that propagates those biases. You end up with a dangerous and self-perpetuating system like this:
1. Data is biased against Black people due to historic racism.
2. AI is built of this biased data.
3. AI results in a biased system that disadvantages Black people.
4. Black people get discriminated against due to the results from the AI.
5. New data is now even more biased against Black people.
6. Go to step 1.