These schoools' associations are all correlations, not guarantees, though.

Howard is a historically black college, which means their student body today is majority black, not 100% black.

Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life.

BYU is a Christian school, but also admits several non-religious people too, and people regularly become apostates to their respective religion.

> Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life.

Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.

If you're going with a probabilistic argument, I think BYU students are more likely to stick with mormonism (let alone Christianity) than Bryn Mawr students with their gender.

That doesn't mean you aren't stereotyping people, though.

Irrespective of the AI discussion (computers don't care if you're stereotyping or not), I really don't think it's stereotyping to assume someone who went to a school which literally only admits women is probably a woman. There's a difference between assuming someone's gender because they're a cheerleader, and assuming because they won the women's cheerleading championship. I also assume that the person who drives a fire truck is a fire fighter.

Right, but the fact the associations are only correlations won't stop the ML model from learning them, if they're present in the training data.

If the training data shows a history of bias against Howard graduates, the model will learn to replicate.

No, again, it is inevitable that outcomes will not be equitable across race. That doesn’t mean all variables are unjustified to use.

Job applications are kind of dumb. The common use case is credit decisions. Surely, for example, you’re ok looking at credit history and income when deciding to make a loan, right? Even though it won’t be equitable outcomes across races, it is still “fair”

The goal of fairness ML models is not necessarily to have equal outcomes across subgroups.

Other goals can be to have equal model accuracy across subgroups (e.g., models has the same precision/auroc/etc for each subgroup, even if the predictions themselves differ across subgroups) or identical distribution of statistical biases across subgroups (e.g., the model makes the same types of mistakes within each subgroup)

While a lot of people will state those are appropriate goals, they are not. They’re really dumb goals. The model being equally bad for each group is not an indicator that you have achieved anything. Most likely you’ve just made your model for white people worse.

Like, let’s look at that for credit decisions. Black folks have a lower income distribution. So it’s like saying the model is ok if it rejects a Black applicant making 80k if it would also reject a White candidate making 100k. That is the insane bullshit you’re effectively supporting by desiring comparable error distributions.

You have to accept that equal outcomes are not just impossible but not desirable. And then you only use fields that people agree are fair to use. If Black people get disproportionately better or worse outcomes it is either because those outcomes make sense given their stats or it’s not ok because if a White dude had the same info, that you agree is fair to use, they would have passed and the underlying reason for the difference is some bullshit other input.

That might mean that the model is a dumb: you must make $100k for this loan (illustrative) which is going to mean a lot of Black folks cannot get it. Well that model is at least “fair”. Accommodations for race should not be coming from an AI model

Agree to disagree

I would love to hear a counter argument that explains why having all models perform equally bad is desirable.