Culture encompasses a range of human behavior. Much of that, though it may not be tasteful to say, is tied to workplace performance. Pressure and desire to achieve, collective/individualist tendency, ethics and morals, conformity and deference to authority; if one is willing to acknowledge that cultures exist and vary, then one cannot deny that there will be strong correlations between culture and fit for particular roles. And as it happens culture correlates with nationality, race, and socioeconomic status - because parents and communities generally pass culture on to children.

At some point in the near future neural nets trained to predict human performance will undoubtedly condition on priors like nationality, race, and socioeconomic status, even indirectly if they are not explicit data points. What then? Will we continue to bend over backwards to deny reality in favor of a false Utopia? What happens when these trends suggest different interventions for medical conditions? Different learning environments? Different reactions to authority and punishment?

Edit: I don't understand the downvotes. Why use data analysis tools if you're just going to ignore the results you don't like? How do you expect to solve problems like inequality if you aren't willing to explore their actual causes? Is there something illogical or factually incorrect in my comment?

Deployed on a wide scale, the filters you describe would indeed constitute systemic racism. However, on an individual basis such practices would be more correctly termed profiling rather than racism since they lack intent. (This depends entirely on your definition of racism, of course, which it turns out can be quite difficult to pin down. [1])

I realize this may seem like splitting hairs since I ultimately agree with your overall point, but often such definitional issues result in a great deal of miscommunication and misunderstanding.

[1] https://slatestarcodex.com/2017/06/21/against-murderism

I would personally argue against any definition of racism that discounts acts based on intent. Your source seems to start out trying to sort-out a kind of binary (trinary) fallacy, where it's arguing that there must be only a single valid definition of what qualifies as racist that can be used in any/all contexts, rather than just acknowledging that there are multiple ways to be racist, with varying degrees of intent/malice/severity/subtly and contextual applicability. It seems to reach that conclusion in the end though with "Overall We Probably Use A Combination Of All Of These, Weighted In Favor Of Definition By Motives." Beyond that, I’d argue that a more limited definition is dangerous as it allows people to argue/believe that a lack of blatantly malicious intent absolves them of the consequences of their acts.

Broadly speaking, yes this would be accurately termed profiling. That is in no way mutually exclusive with being racist, given either of our definitions of racism. The developer of an AI profiling algorithm must make choices about what data is provided to that system, and others must make a choice to act based on the outputs of that system. The choice to include racial or socioeconomic criteria (or criteria that are proxies for these) as training data for a given model, and act on its outputs in ways that could contribute to systemic racism, could be done without malice or could be driven by blatantly racists motives. We just end up back at the same argument about the definition of racism.

And again, I’d argue that the more limited definition is dangerous as it allows exactly what wlesieutre meant with “we've decided to deflect blame.” Allowing an act that leads to systemic racism to not qualify as racism without a clear blatant intent only serves to excuse those developing/using such systems for the consequences of their choices. Additionally, the narrower definition could actually discourage consideration of such consequences, as acting without any thought or consideration at all is acting without intent. Thus, willful or feigned ignorance becomes a defense against the furtherance of systemic racism.

I suspect in many cases, folks who implement racist systems simply do not care whether their system is racist or not. They want to optimize/automate whatever it is that they are about, and if the result is unfair, well, that's just someone else's problem.

This doesn't feel like an argument in good faith because you both seem to already have strong beliefs about what the "actual causes" are, and also place a lot of words in the mouths of those who might disagree with you based on an uncharitable interpretation of their motives.

Consider how you lump in "nationality, race, and socioeconomic status" all of the same category. It seems you've already decided that those are all root causes, and anyone who disagrees is just denying reality.

One might say "I want to be able to identify interventions to counteract negative effects of [socioeconomic status, home life, etc] in a specific and personalized way to improve opportunities." That would be an interesting use of computational power, beyond what we could've accomplished without fancy computers.

Or you could design a model that would just mimic simple naive stats and reinforce your beliefs on race and culture, while not actually be digging into precise, specific, details. Yes, your simple model will label groups that have suffered historical and ongoing discrimination and disadvantages as less likely to succeed if you don't fully model the compounding and generational effects of that. That's neither interesting nor compelling. People are extremely complicated and yet you've concluded that it's all down to a few blunt factors. Ok.

That last one in particular has serious implications relating to biases and socioeconomic factors.

> Will we continue to bend over backwards to deny reality in favor of a false Utopia?

That would be a rather extreme position. Why do we have to choose between ignoring things that are broken versus ignoring reality itself? Why not attempt to work towards an ideal, even if that ultimately costs our society some small amount of efficiency on the whole?

Yeah you do.