In fact, an AI trained on aggregate data to probabilistically infer characteristics about individuals is _literally a stereotyping machine_.
If people are upset that their stereotyping machine stereotypes people, they probably didn't fully understand what they were doing when they built it, because this is not a design flaw -- it's the design.
Then these people should argue against AI instead of swinging the racism cudgel.
More concrete: government AI shouldn't use things like names, zipcode demographics (at least those strongly to those characteristics we think discrimination = racism) and pictures of humans in their models. Why? Because it's pretty much impossible to control your model for racist tendencies one you start there. It's in the ethics of the creator of the model to point that out and just don't do it. If you do, and all people whose name start with an M (for Mohammed) get a different category, racist is the right term IMHO.
Quite a tautology.
Video: https://www.youtube.com/watch?v=Zn7oWIhFffs
Slides: https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...
It is a kind of optimality condition on all three goals.
The robustness additionally means that should conditions change, the algorithm usually will become better not worse and should a degradation still happen, it will be graceful and not catastrophical.
It's a hard and open problem in ML and especially ANN, design of robust solutions in the space. Most have really bad problems with it even when debiased.
Not that I mind that too much!
This is likely due to an acknowledgment of the limits of human models to account for the full context surrounding correlated-but-non-causal classifications, such that conclusions drawn from them can have unforeseen or highly detrimental ramifications.
Speaking to race in America specifically, the schema through which we judge people are highly susceptible to bias from the white supremacist bent of historical education and general discourse. This is how you end up with cycles like those within the justice system (pushed in part by sentencing software), wherein black defendents are assumed to have a higher likelihood of re-offending, therefore increasing the likelihood of any given black defendent not receiving bond or having a lengthy sentence if convicted. After all, blackness correlates with recidivism. Lying outside this correlative relationship is the likely causal relationships of longer stays in jail and lack of access to employment opportunities, which disproportionately affect black people, causing higher rates of recidivism, regardless of race.
You can still have enhanced vigilance without enhanced annoyance and mistakes.
There's often a superior choice lurking that nobody is thinking about, sometimes expensive, sometimes not, seemingly unrelated to such optimization. This is why ML is not intelligence, it cannot find new solutions you're not already looking for.
The main problem of the judicial and police system is it tries to be procedurally fair and still fails at it anyway.
Ironically, the only places where it's legally prohibited or frowned upon to use these heuristic techniques are situations that people have arbitrarily (heuristically or conveniently) decided.
For example: It's "not fair" to hire someone because they're white (and consequently have a higher chance of being wealthy and hence a higher change of being educated.)
But it's "fair" to choose a love partner based on their height, their waist-to-hip ratio, their weight (and hence having a higher chance of giving birth to healthy offspring, better physical protection, etc.).
Maybe it's hypocritical, and I don't know if that's a good thing or not. Maybe being hypocritical helps us survive.
This applies to companies of any size.
Sure, some people in some countries have tried (sometimes successfully) to undermine this principle, but it's akin to forcing people to marry in groups.
Societal restrictions on contracts are a way to balance this. Virtually all societies have some form of this, outside utopian ultra-liberal hellholes.
I get what you're saying, but no one moves jobs every week. The sunk costs of switching employment are significant for the employee, less so for the company.
To a lesser degree, the same goes with vacation time and other benefits. At least in the US, anyway. This is why having some of this stuff coded into law and decoupled from employment takes some power away from employers.
Of course, this is a point of view and not everyone agrees with me, but to me it appears that for a chunk of the population the available jobs do not pay well enough to meet a certain standard of living.
That is definitely true, but it's also pretty much the exact opposite of "always".
What? Can you explain the reasoning behind this statement?
Of course, that scenario happens only in extreme edge cases. Even in a booming economy with a shortage of workers, John Doe is not going to be pursued aggressively to fill the Senior Marketing Manager role.
This makes intuitive sense: employers have a ton of money, so people come to them.
Right now I have a client who needs to hire truck drivers and can't do it fast enough. I asked him what he'd done to make his company the most attractive (pay, technology, perks, etc.). He said he's done nothing.
That's exactly what recruiters and headhunters do. Aggressively nails it.
Most people will never be headhunted.
A candidate has multiple job offers, and decides which one to take.
The company interviews multiple people for a single position, rejecting the others.
There are only a small number of sectors where the first situation is reasonably possible, a lot of us on here are incredibly fortunate that engineering happens to be one of them. For the majority of the job market (by volume of people rather than volume of money), it takes people attempt after attempt to get a job. They don't get to choose between multiple offers, they have to take the first thing that will allow them to pay the rent, and then they have to hold on to it.
Europe has the concept of indirect discrimination, which could make that illegal, certainly things less central to the role could amount to indirect discrimination.
"are situations that people have arbitrarily (heuristically or conveniently) decided"
The Holocaust was not convenient, even to those who were for it. Slavery was convenient for those who benefited from it, but not those who suffered under it. Over the last 200 years racial categories lead to many millions of dead. This is not merely a matter of convenience.
Violence and coercion don't logically follow from racial differentiation. One may point out the differences between populations of different races, but that wouldn't justify attacking any individual from those populations.
My comment is framed inside that basic (and obvious) principle. Choosing a partner or an employee is not a violent or coercive act.
That sounds like the wrong basis for calling it extreme. It's not at all extreme to say that classifying an interview candidate based on correlated but non-causal characteristics is wrong, regardless of the statistical significance of those correlations.
https://www.fool.com/investing/2019/05/15/rihannas-fenty-is-...
And this is actually the rational thing to do. Reason: There are two potential errors involved here: labeling an innocent person a criminal (false positive) and labeling a criminal as innocent (false negative). The key is to realize that the cost of these two errors is not the same. For instance, treating an innocent person as a criminal could be much more expensive for the person and society than not detecting a criminal with a given classifier. For that very reason, we have the presumption of innocence as a principle in law. As a consequence, you don't want to select a classifier based on just the rate of errors overall, but you want to incorporate some kind of loss function that minimizes the cost for individuals and society. Under that loss function, the best classifier may actually be wrong more often than some other classifier.
However, had this been done to my daughter and the same result obtained (in fact the result was "blue" but there you go) by someone assessing her potential as a candidate for university - well there's damage.
For example: It may be politically correct to drive down a bad neighborhood in the middle of the night, but it isn't logically correct.
But note the implicit bias in your own comment: you assume yourself and all the readers of the comment are not people who live in bad neighborhoods.
that assumption results simply from the need for 'bad neighborhood' to be a negative scoring action.
if you oversimplify it to get rid of 'implicit bias' (which I don't agree exists in the example), the results turn into near-meaningless babble.
"For example: It may be politically correct to do something that ignores statistical dangers in favor of the promise of human goodness, which may result in the possibility for more personal endangerment than other choices, but it isn't logically sound to ignore such statistics for the hope of a less biased personal experience."
The example requires the person driving to be detached from the bad neighborhood that they have a choice to drive through. How that isn't an obvious requirement for the example to have merit is beyond me.
After all, you don't really need a law telling companies they aren't allowed to hire infants as senior officers - it's already not in the company's interest to do so.
However, when there is a logically correct but politically incorrect decision that the company could make, it is now that you need laws to prevent the company from taking that choice.
Of course, as the weight of an institution's decisions goes down, so does the need to police its actions. In particular, it is rarely necessary to prevent an individual person from acting on their biases.
Applying this to your example, if we had an AI that should suggest your best route home, and it avoided a short route through a bad neighborhood, that is likely ok. However, if a municipality used the same AI to decide where to prioritize changing street lights, that should be prohibited.
Sorry, but it's not a decision. Science has found repeatedly that using outward characteristics does not work as a good classification measure. Society simply enforces not making bad judgements. See:
Phrenology, Racism -- "appearance implies certain things about someone's character and intelligence" (https://en.wikipedia.org/wiki/Phrenology)
Sex -- "A person's gender presentation determines their gender or sex" (https://en.wikipedia.org/wiki/Transgender_history), "A person's genitalia determines their chromosones", "A person's chromosones determine their sex and their ability to give birth" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2190741/).
Wealth -- "That person is wearing bad clothing/living an unassuming life therefore they must be poor" (https://www.npr.org/2018/12/29/680883772/social-worker-led-f...)
Intelligence -- "This person scores badly on intelligence tests therefore they must not be worthwhile" (https://en.wikipedia.org/wiki/Richard_Feynman, https://www.psychologytoday.com/gb/blog/sudden-genius/201101...), "This person appears to be unskilled or incapable of many tasks therefore they must be totally unskilled" (http://en.wikipedia.org/wiki/Savant_syndrome)
Science has found that, on average, it works [1]. Of course there will be many cases where it fails - that's how statistical inference works. Whether that makes it a 'good' measure, by whatever standard, is a different question. But there is no doubt information can be inferred from appearance.
[1] http://www.spsp.org/news-center/blog/stereotype-accuracy-res...
Lol, what? Generically, that statement is almost certainly false more often than true in general in science. But I believe you are restricting yourself to social psychology?
You then presented a long list of examples taken from the tails of certain distributions to refute an argument that said distribution exists and has an average? I didn't even name any particular distributions. You're thinking appears flawed and emotionally driven, and most unfortunately, that's the type of thinking that will lead you to building biased systems.
Here's the point you missed the first time around: There are going to be outwardly visible characteristics that ARE correlated with some factor of interest, to the extent that training a machine learning algorithm based on a cost function that uses predictive accuracy alone WILL result in a system that assigns what society would consider an inappropriate importance placed on non-causal but correlated parameters.
Here's a real world example that might help you understand why this is important: (Data taken from: https://en.wikipedia.org/wiki/Incarceration_in_the_United_St...) Because blacks are over-represented in the US criminal justice system (40% of the prison population vs 13% of the population) and because part of what defines "black" is the outward appearance of certain facial features, a facial-recognition algorithm which is trained to recognize criminals, with a cost function based on prediction accuracy alone, and facial features as input parameters is going to have false positives that over-represent blacks. Does that sound like something you want? Because denying the underlying distributions is going to lead to exactly that.
It's very important to consider this when you develop a training set, for fucks sake. It might work something like this: Take 100 innocent people's faces at random. (On average it will have only 13 blacks) Then take 100 random criminal faces from inmates. (On average it will have 40 blacks.)
Then mix up the groups into your training set and assign a prediction score 1 or 0 depending on whether or not your classifier has correctly predicted whether or not a face was in the criminal group. Then, based on no other feature than race, your neural net can get better performance based solely on guessing more often that black people are criminals. That's not a good thing.
Do you get it now? The likelihood of being falsely identified as a criminal is greater based on the non-causal but correlated variable of being black. And this has happened several times already! You can't keep pushing this narrative that neglects the underlying statistics because of your beliefs, or people will keep making racist systems.
Trying to identify potential for criminality based on looks may well work, identifying and using the underlying racial biases. And if those systems are used, where people identified by these systems are more likely to be identified as criminals and investigated, we end up with a feedback loop and, over time, the racial biases in a society that uses the systems will get worse. More black criminals will be caught, as they are more likely to be suspected as criminal, making the racial biases worse for blacks. While the opposite happening for white faces.
Similar social evolution would happen if you pre-screened job candidates, over time magnifying existing gender and racial biases. I've seen in some cultures it is required to add photographs to job applications, but I think it a good thing that this practice is discouraged in western countries.
Say in your example of criminality, if black people were more predicted by the classifier, it isn't racist if it accurately reflects the base distribution. There's a line to be crossed here, and to me it's if that application starts to significantly (where the boundary lies here is up for debate), and directly affect the non-criminal portion. I'd say that if the misclassification error is similar across racial lines, then there is no issue.
Additionally, I don't quite agree with the "making racial biases worse" argument either. The way I see it, we already use racial heuristics in law enforcement. With automated, replicable tasks, we can at least quantify the degree of bias and correct for such.
Most people would probably opt for utility mixed with representational fairness of some degree even when it means law applies to some degree differently to groups and special cases.
Justice is not fairness. It has an institution called motive. (Which is often overly simplified or ignored.)
It incorporates elements of fairness. And it is hard to train. Not everyone can be Solomon, for example.
https://www.newscientist.com/article/2114900-concerns-as-fac...
In short, being humane. If a certain degree of racism or stereotyping is necessary for that goal (definitely not 100%, but something low), then so be it.
Currently multiple social system in the USA are considered much too racist.
What underlines it is seemingly a lack of understanding of statistics, becausee the most "examples", if they can even be called that are from the realm of non-common outlier cases. That's not what statistics operates on mostly - it operates on common cases that cover the largest part of a distribution. Not outlier cases on the ends of a distribution.
In every case you gave an example of some uncommon exception, as if that would somehow be an argument for the rule existing?
Thing is, that's not even true in that it doesn't fully acknowledge the problem. You often CAN tell information from people's outward appearance, albeit probabilistically. Therein lies the problem: you can very easily train an algorithm to be maximally right according to your cost function, but end up biased because the underlying distributions aren't the same between groups.
The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption.