Credit risk, f(X), is an unknown population function that needs to be estimated using observed data X.
If including tallness into X improves our estimate of f(X), then we've gotten a better model.
You've asserted that X should only contain an individual's past actions instead of their inherent traits such as tallness. This may satisfy certain political objectives, and that's fine if we're being upfront about the underlying motivation, but from an ML perspective your prescription doesn't make much sense unless you have some prior knowledge about the function f(X) that tells you that tallness both isn't relevant and isn't acting as an instrumental variable for some other missing feature.
Unless you have such domain knowledge, you've little business asserting what are appropriate features to use in order to improve model quality.
If you think it is, that's fine. We might as well just taking the same approach to crime, and start locking individuals up or not extending job offers, NOT because they've done a single thing wrong, but simply because they're statistically more likely to.
The main difference is that people have right to trial and to be considered innocent until proven guilty. But there is no 'right to credit'. Credit is fundamentally two-party contract.
Also there is shared limit to risk, forcing creditors to take more risk with some people means they may not take that risk in other cases (not giving credit to someone who would be marked lower risk with more informed decision) or forcing them to raise credit cost to everybody.
The crime analogy is inappropriate. Insurance is a private voluntary arrangement between two consenting entities. Convictions on the other hand are an involuntary imposition on an unwilling party.
One could make the argument that allowing inherent traits in the pricing of insurance is the less authoritarian and more utilitarian option, since it is less forceful state interference in private business and leads to more accurate claims pricing and less subsidisation of insurance for person A by person B. Your prison sentencing analogy on the other hand implies more force, which is why I don't view it as a valid analogy.
The ethical argument can go either way depending on the axioms we pick a priori. If we pick Libertarian deontological axioms, then the ethical choice is to allow inherent traits into the model. If we pick racial equity deontological axioms, then we get another conclusion.
Which is an important thing that isn’t math.
Consider a sample size of 10/1million with height value of 7m. And somehow 7/10 had poor ability to repay loans. With such a small sample size of this relevant factor be a good thing to rely on?
Would you be OK if banks started charging you 4% more interest because say you have irish heritage? Your behaviour hasn't changed and nobody can explain why it matters that your father was irish, but the models determined it's statistically significant,
What if at some point 30 years down the line it's determined it wasn't even related to irish heritage. There was just coincidentally long running extreme predatory lending by banks in cities with large irish populations.
Would you still feel there's nothing wrong with it?
What if, one bank started doing it because they used this model. But another bank doesn't, because they don't believe this model?
If you were an irish person, you would just move banks. And if it turns out, indeed, that the first bank was over-charging the 4% (ie., no irish person defaults due to their being irish), then they've lost business for no reason, and the 2nd bank got more business.
However, if it turns out that being irish _does_ indeed cause you to default more, then the bank charging an extra 4% was correct. The 2nd bank, upon finding proof of this, would also start charging an extra 4% (otherwise, they'd be losing out money to the first bank).
So in the end, a bank can only charge the most appropriate interest rate for the market to bear.
Most banks just copy each other.
Credit worthiness should be because of an individuals past actions and not some attribute they may have been born with. Why should a tall individual who's never missed a payment have some invisible penalization applied because some other tall people are worse at re-paying loans.
Don't you see the issue here? You're penalizing individuals not based on their own measurable behavioral signals, but simply due to some physical trait they share with others. Not to mention, we wouldn't even understand why there's a correlation in the first place. Maybe tall people are 30% likely to have some unknown gene, why should the other 70% of tall people who don't have the gene be treated the same?