I also think that it is possible that the model learned that information from too small of a data sample. What is a good data sample for every such feature in a relatively balanced manner is really difficult to build a dataset from.

Consider a sample size of 10/1million with height value of 7m. And somehow 7/10 had poor ability to repay loans. With such a small sample size of this relevant factor be a good thing to rely on?