How about if they wrote software with a single criteria for loans, and that criteria was "don't lend to black people"?
Right, we know that's not OK intuitively. What about an algorithm that automatically declines all applications from people who went to historically black colleges?
So then, where exactly do you draw the line? If said algorithm is opaque does that make it OK? What if it wasn't intentional, but the algorithm just kind of turned out that way by machine learning? Is that better somehow? And how can we tell, and how can we hold the bank accountable for those decisions?
One answer to "where do we draw the line" is to say that we only make credit decisions based on history and experience with actual credit. Things like payment histories and so on, and that the both the criteria and the data used to make those decisions must be transparent, and consumers have the right to challenge it.
This, in fact, is the system we currently have. There are reasons for that.
I can't think of any advantages a racist bank would have, can you? The approach by this lender is rather to dig deeper into individuals and give them an opportunity based on personal data. It seems just as likely that resulting correlations could help any given race.
I believe algorithmic attempts to understand people are a form of intelligence to support, even if they discover things we might not like or make mistakes along the way. Should they find uncomfortable truths we can own up, have a laugh and grow; it will be a lot easier than fighting facts.
Prejudice is literally pre-judging, making assertions based on fear without knowing the facts. Perhaps it's the algorithms themselves that are in need of protection here.
Banks can, which is why redlining was a thing: https://en.wikipedia.org/wiki/Redlining
One of the problems comes from the kind of virtual redlining you can get via closely correlated metrics that serve as a proxy for problematic criteria. For example, someone's preferences in music or entertainment or food purchasing could closely correlate to their ethnicity, which in turn could correlate to lowered access to capital due to historic racial discrimination. You may be telling yourself you're writing an impartial algorithm, but it's not impartial, it's just opaque. How do we know that the "prepaid cellular" metric isn't just a proxy for "black people" or "recent immigrants" for example?
Likewise, you have the other problem, which is trusting the data in the first place. We have some stringent laws in this country about the ability for a consumer to see and correct data that leads to negative lending decisions being made about them. Perhaps its interesting that "giving up a prepaid wireless number" is a really bad data point when deciding to lend me money. But what if I never actually had a prepaid phone at all, and some database somewhere that I can't see says that I did? That's not only a problem it's actually illegal.
The article makes a show of taking a critical eye towards this method of evaluating creditworthiness, but somehow doesn't raise two of the really serious Achilles heels that are baked into all such schemes and have been problematic for awhile.