Thing is, that's not even true in that it doesn't fully acknowledge the problem. You often CAN tell information from people's outward appearance, albeit probabilistically. Therein lies the problem: you can very easily train an algorithm to be maximally right according to your cost function, but end up biased because the underlying distributions aren't the same between groups.

The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption.

In fact, an AI trained on aggregate data to probabilistically infer characteristics about individuals is _literally a stereotyping machine_.

If people are upset that their stereotyping machine stereotypes people, they probably didn't fully understand what they were doing when they built it, because this is not a design flaw -- it's the design.

Then these people should argue against AI instead of swinging the racism cudgel.

More concrete: government AI shouldn't use things like names, zipcode demographics (at least those strongly to those characteristics we think discrimination = racism) and pictures of humans in their models. Why? Because it's pretty much impossible to control your model for racist tendencies one you start there. It's in the ethics of the creator of the model to point that out and just don't do it. If you do, and all people whose name start with an M (for Mohammed) get a different category, racist is the right term IMHO.

Quite a tautology.

Video: https://www.youtube.com/watch?v=Zn7oWIhFffs

Slides: https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...

It is a kind of optimality condition on all three goals.

The robustness additionally means that should conditions change, the algorithm usually will become better not worse and should a degradation still happen, it will be graceful and not catastrophical.

It's a hard and open problem in ML and especially ANN, design of robust solutions in the space. Most have really bad problems with it even when debiased.

Not that I mind that too much!

This is likely due to an acknowledgment of the limits of human models to account for the full context surrounding correlated-but-non-causal classifications, such that conclusions drawn from them can have unforeseen or highly detrimental ramifications.

Speaking to race in America specifically, the schema through which we judge people are highly susceptible to bias from the white supremacist bent of historical education and general discourse. This is how you end up with cycles like those within the justice system (pushed in part by sentencing software), wherein black defendents are assumed to have a higher likelihood of re-offending, therefore increasing the likelihood of any given black defendent not receiving bond or having a lengthy sentence if convicted. After all, blackness correlates with recidivism. Lying outside this correlative relationship is the likely causal relationships of longer stays in jail and lack of access to employment opportunities, which disproportionately affect black people, causing higher rates of recidivism, regardless of race.

You can still have enhanced vigilance without enhanced annoyance and mistakes.

There's often a superior choice lurking that nobody is thinking about, sometimes expensive, sometimes not, seemingly unrelated to such optimization. This is why ML is not intelligence, it cannot find new solutions you're not already looking for.

The main problem of the judicial and police system is it tries to be procedurally fair and still fails at it anyway.

Ironically, the only places where it's legally prohibited or frowned upon to use these heuristic techniques are situations that people have arbitrarily (heuristically or conveniently) decided.

For example: It's "not fair" to hire someone because they're white (and consequently have a higher chance of being wealthy and hence a higher change of being educated.)

But it's "fair" to choose a love partner based on their height, their waist-to-hip ratio, their weight (and hence having a higher chance of giving birth to healthy offspring, better physical protection, etc.).

Maybe it's hypocritical, and I don't know if that's a good thing or not. Maybe being hypocritical helps us survive.

This applies to companies of any size.

Sure, some people in some countries have tried (sometimes successfully) to undermine this principle, but it's akin to forcing people to marry in groups.

Societal restrictions on contracts are a way to balance this. Virtually all societies have some form of this, outside utopian ultra-liberal hellholes.

I get what you're saying, but no one moves jobs every week. The sunk costs of switching employment are significant for the employee, less so for the company.

To a lesser degree, the same goes with vacation time and other benefits. At least in the US, anyway. This is why having some of this stuff coded into law and decoupled from employment takes some power away from employers.

Of course, this is a point of view and not everyone agrees with me, but to me it appears that for a chunk of the population the available jobs do not pay well enough to meet a certain standard of living.

That is definitely true, but it's also pretty much the exact opposite of "always".

What? Can you explain the reasoning behind this statement?

Of course, that scenario happens only in extreme edge cases. Even in a booming economy with a shortage of workers, John Doe is not going to be pursued aggressively to fill the Senior Marketing Manager role.

This makes intuitive sense: employers have a ton of money, so people come to them.

Right now I have a client who needs to hire truck drivers and can't do it fast enough. I asked him what he'd done to make his company the most attractive (pay, technology, perks, etc.). He said he's done nothing.

That's exactly what recruiters and headhunters do. Aggressively nails it.

Most people will never be headhunted.

A candidate has multiple job offers, and decides which one to take.

The company interviews multiple people for a single position, rejecting the others.

There are only a small number of sectors where the first situation is reasonably possible, a lot of us on here are incredibly fortunate that engineering happens to be one of them. For the majority of the job market (by volume of people rather than volume of money), it takes people attempt after attempt to get a job. They don't get to choose between multiple offers, they have to take the first thing that will allow them to pay the rent, and then they have to hold on to it.

Europe has the concept of indirect discrimination, which could make that illegal, certainly things less central to the role could amount to indirect discrimination.

"are situations that people have arbitrarily (heuristically or conveniently) decided"

The Holocaust was not convenient, even to those who were for it. Slavery was convenient for those who benefited from it, but not those who suffered under it. Over the last 200 years racial categories lead to many millions of dead. This is not merely a matter of convenience.

Violence and coercion don't logically follow from racial differentiation. One may point out the differences between populations of different races, but that wouldn't justify attacking any individual from those populations.

My comment is framed inside that basic (and obvious) principle. Choosing a partner or an employee is not a violent or coercive act.

That sounds like the wrong basis for calling it extreme. It's not at all extreme to say that classifying an interview candidate based on correlated but non-causal characteristics is wrong, regardless of the statistical significance of those correlations.

https://www.fool.com/investing/2019/05/15/rihannas-fenty-is-...

And this is actually the rational thing to do. Reason: There are two potential errors involved here: labeling an innocent person a criminal (false positive) and labeling a criminal as innocent (false negative). The key is to realize that the cost of these two errors is not the same. For instance, treating an innocent person as a criminal could be much more expensive for the person and society than not detecting a criminal with a given classifier. For that very reason, we have the presumption of innocence as a principle in law. As a consequence, you don't want to select a classifier based on just the rate of errors overall, but you want to incorporate some kind of loss function that minimizes the cost for individuals and society. Under that loss function, the best classifier may actually be wrong more often than some other classifier.

However, had this been done to my daughter and the same result obtained (in fact the result was "blue" but there you go) by someone assessing her potential as a candidate for university - well there's damage.

For example: It may be politically correct to drive down a bad neighborhood in the middle of the night, but it isn't logically correct.

But note the implicit bias in your own comment: you assume yourself and all the readers of the comment are not people who live in bad neighborhoods.

that assumption results simply from the need for 'bad neighborhood' to be a negative scoring action.

if you oversimplify it to get rid of 'implicit bias' (which I don't agree exists in the example), the results turn into near-meaningless babble.

"For example: It may be politically correct to do something that ignores statistical dangers in favor of the promise of human goodness, which may result in the possibility for more personal endangerment than other choices, but it isn't logically sound to ignore such statistics for the hope of a less biased personal experience."

The example requires the person driving to be detached from the bad neighborhood that they have a choice to drive through. How that isn't an obvious requirement for the example to have merit is beyond me.

After all, you don't really need a law telling companies they aren't allowed to hire infants as senior officers - it's already not in the company's interest to do so.

However, when there is a logically correct but politically incorrect decision that the company could make, it is now that you need laws to prevent the company from taking that choice.

Of course, as the weight of an institution's decisions goes down, so does the need to police its actions. In particular, it is rarely necessary to prevent an individual person from acting on their biases.

Applying this to your example, if we had an AI that should suggest your best route home, and it avoided a short route through a bad neighborhood, that is likely ok. However, if a municipality used the same AI to decide where to prioritize changing street lights, that should be prohibited.

Sorry, but it's not a decision. Science has found repeatedly that using outward characteristics does not work as a good classification measure. Society simply enforces not making bad judgements. See:

Phrenology, Racism -- "appearance implies certain things about someone's character and intelligence" (https://en.wikipedia.org/wiki/Phrenology)

Sex -- "A person's gender presentation determines their gender or sex" (https://en.wikipedia.org/wiki/Transgender_history), "A person's genitalia determines their chromosones", "A person's chromosones determine their sex and their ability to give birth" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2190741/).

Wealth -- "That person is wearing bad clothing/living an unassuming life therefore they must be poor" (https://www.npr.org/2018/12/29/680883772/social-worker-led-f...)

Intelligence -- "This person scores badly on intelligence tests therefore they must not be worthwhile" (https://en.wikipedia.org/wiki/Richard_Feynman, https://www.psychologytoday.com/gb/blog/sudden-genius/201101...), "This person appears to be unskilled or incapable of many tasks therefore they must be totally unskilled" (http://en.wikipedia.org/wiki/Savant_syndrome)

Science has found that, on average, it works [1]. Of course there will be many cases where it fails - that's how statistical inference works. Whether that makes it a 'good' measure, by whatever standard, is a different question. But there is no doubt information can be inferred from appearance.

[1] http://www.spsp.org/news-center/blog/stereotype-accuracy-res...

Lol, what? Generically, that statement is almost certainly false more often than true in general in science. But I believe you are restricting yourself to social psychology?

You then presented a long list of examples taken from the tails of certain distributions to refute an argument that said distribution exists and has an average? I didn't even name any particular distributions. You're thinking appears flawed and emotionally driven, and most unfortunately, that's the type of thinking that will lead you to building biased systems.

Here's the point you missed the first time around: There are going to be outwardly visible characteristics that ARE correlated with some factor of interest, to the extent that training a machine learning algorithm based on a cost function that uses predictive accuracy alone WILL result in a system that assigns what society would consider an inappropriate importance placed on non-causal but correlated parameters.

Here's a real world example that might help you understand why this is important: (Data taken from: https://en.wikipedia.org/wiki/Incarceration_in_the_United_St...) Because blacks are over-represented in the US criminal justice system (40% of the prison population vs 13% of the population) and because part of what defines "black" is the outward appearance of certain facial features, a facial-recognition algorithm which is trained to recognize criminals, with a cost function based on prediction accuracy alone, and facial features as input parameters is going to have false positives that over-represent blacks. Does that sound like something you want? Because denying the underlying distributions is going to lead to exactly that.

It's very important to consider this when you develop a training set, for fucks sake. It might work something like this: Take 100 innocent people's faces at random. (On average it will have only 13 blacks) Then take 100 random criminal faces from inmates. (On average it will have 40 blacks.)

Then mix up the groups into your training set and assign a prediction score 1 or 0 depending on whether or not your classifier has correctly predicted whether or not a face was in the criminal group. Then, based on no other feature than race, your neural net can get better performance based solely on guessing more often that black people are criminals. That's not a good thing.

Do you get it now? The likelihood of being falsely identified as a criminal is greater based on the non-causal but correlated variable of being black. And this has happened several times already! You can't keep pushing this narrative that neglects the underlying statistics because of your beliefs, or people will keep making racist systems.

Trying to identify potential for criminality based on looks may well work, identifying and using the underlying racial biases. And if those systems are used, where people identified by these systems are more likely to be identified as criminals and investigated, we end up with a feedback loop and, over time, the racial biases in a society that uses the systems will get worse. More black criminals will be caught, as they are more likely to be suspected as criminal, making the racial biases worse for blacks. While the opposite happening for white faces.

Similar social evolution would happen if you pre-screened job candidates, over time magnifying existing gender and racial biases. I've seen in some cultures it is required to add photographs to job applications, but I think it a good thing that this practice is discouraged in western countries.

Say in your example of criminality, if black people were more predicted by the classifier, it isn't racist if it accurately reflects the base distribution. There's a line to be crossed here, and to me it's if that application starts to significantly (where the boundary lies here is up for debate), and directly affect the non-criminal portion. I'd say that if the misclassification error is similar across racial lines, then there is no issue.

Additionally, I don't quite agree with the "making racial biases worse" argument either. The way I see it, we already use racial heuristics in law enforcement. With automated, replicable tasks, we can at least quantify the degree of bias and correct for such.

Most people would probably opt for utility mixed with representational fairness of some degree even when it means law applies to some degree differently to groups and special cases.

Justice is not fairness. It has an institution called motive. (Which is often overly simplified or ignored.)

It incorporates elements of fairness. And it is hard to train. Not everyone can be Solomon, for example.

https://www.newscientist.com/article/2114900-concerns-as-fac...

In short, being humane. If a certain degree of racism or stereotyping is necessary for that goal (definitely not 100%, but something low), then so be it.

Currently multiple social system in the USA are considered much too racist.

What underlines it is seemingly a lack of understanding of statistics, becausee the most "examples", if they can even be called that are from the realm of non-common outlier cases. That's not what statistics operates on mostly - it operates on common cases that cover the largest part of a distribution. Not outlier cases on the ends of a distribution.

In every case you gave an example of some uncommon exception, as if that would somehow be an argument for the rule existing?

This. Some Childhood development studies have born out that children preferentially treat their own 'ingroup' preferentially over those of an 'outgroup' [1]. This is before they really are "impressionable" as I understand it, so it shows something of an inbuilt mechanism. Though they made this distinction:

“Racism connotes hostility and that’s not what we studied. What the study does show is that babies use basic distinctions, including race, to start to cleave the world apart by groups of what they are and aren’t a part of.” [2]

1 - https://www.frontiersin.org/articles/10.3389/fpsyg.2018.0175...

2 - https://www.telegraph.co.uk/news/science/science-news/107705...

At least some of the people conducting the studies on children are guilty of child abuse, for they tried to correct the behavior of infants with their infantile theories.

It might make interesting reading to learn about people who don't have an ingroup or who don't prefer their ingroup.

Or then again, it might just be depressing.

This was the classical experiment I remember: https://en.wikipedia.org/wiki/Realistic_conflict_theory#Robb....

https://en.wikipedia.org/wiki/Thoughtcrime#Crimestop

We basically need this in software so that AI researchers can stop getting lambasted with claims of racism

Empathy is an important part of resolution, as much as that sucks.

They give everyone tests which reveal that primitive part of your brain. Essentially they want to shock you by forcing you to fail. It is almost like they are trying to shame people, which is a terrible way to teach.

There’s nothing to be ashamed of when it comes to having bias: women think other women are less competent than their male coworkers, trans women struggle with thinking of other trans women as men, black men find other black men threatening. Your first thought is the one you’ve been conditioned to think and is broadly speaking shared by everyone. It’s when you don’t stop to have the second thought — your own thought that’s the problem.

“No, that’s silly. I’ve seen her work — she absolutely knows what she’s talking about.”

“She’s a woman. Full stop.”

“That guy is just minding his own business and given zero actual signs of being a threat.”

You won’t hear much about it though because: 1) it’s a sensitive topic that is easily misunderstood by those who aren’t researchers 2) when everyone else is trying to scrub racism out of their models for politically correct reasons, having a “racist” AI can actually produce a competitive advantage in some industries, and it can be a difficult advantage for competitors to get similar performance if they don’t allow for natural racism to emerge.

In short, it’s not a big deal, humans have some amount of racism whether they admit it or not. What matters is that you treat people equally and without prejudice, regardless what you may think of their race. Judgements about a group and judgements about an individual are two different things.

Racist results don’t even need to be about negative things, it could be as simple as “a person of this race prefers this kind of food over that one”

If my classifier predicts that an Asian guy would probably eat rice, if it's accurate, is it racist?

The issue I'd have here isn't racism, it's when race dominates other features resulting in, in this case, poor recommendation quality/variety for Asians who prefer non-rice dishes or vice versa.

Well there’s all kinds of mimicry in nature to exploit exactly this assumption.

Isn't that just a bug in the English language though? English very rarely employs variations of the same word based on gender - but that's not true of many other languages. If the labeling was done in say Polish or German suddenly it wouldn't be biased at all, a male doctor would be labeled Lekarz/Artz while a female one would be labeled Lekarka/Ärztin - it's just what it is, no bias here.

Our minds are literally categorical machines--in order to fight entropy we find stable states through classification of the world into categories. So literally any form of thought is a form of discrimination between many categories. Every word can be thought of as a category.

Absolutely. The mark of intelligence is to recognize them and correct for it.

And the people who just permanently slip up, and never correct themselves, and somehow always make the same snap judgments? 100% racist.

I mean it doesn’t seem like people tended to label some races with negative labels, just that they labeled their race at all, and didn’t do that for white people. At least in the example.

I don’t think it is useful to lump these two concepts together.

You can't understand very much at all just based on how things look. That holds for humans and inhumans alike.

When an arborist "looks" at a tree, they identify the kind of tree it is, and that connects to all of the research they know about that species of tree. It's reasonable to suppose that a new instance of a known kind of tree is going to have properties in common with all the other known instances.

Your point about how a specialist can connect their knowledge to what they see is true, but not relevant to the point at hand. If I was your therapist, and had studied and understood you thoroughly, I might be able to assess your mental state at a glance in most scenarios. That doesn't deny you have a detailed inner life anymore than that an arborist might generally understand a tree does not deny the complex life of the tree.

Yes you can do that. To put it extremely simply: define what "friendly", "sick", and "lazy" means in terms of behavior, then observe the dog's behaviors.

Edit: sleeping is a behavior.

?? Obviously you can. Why do you think you can't?

you can't identify every friendly dog just by looking at it, you'll identify dogs that are also acting friendly and doing things people have found preceded positive experiences.

you can't identify every sick dog just by looking at it, otherwise the vet would never find additional issues.

and so on

Yes, it's wrong to assume that you can tell everything about that dog by looking at it - but it's so much more uncomparably wrong to consider that you can tell nothing about friendly or sick dogs based on looking at them!

Observating the behaviors and dog breeds correlating with previous friendly and unfriendly experiences is a very useful, somewhat reliable predictor of how likely this particular dog is to be (un)friendly. Sure, if you know this particular dog, then that should supercede any group information, but if not, then that's all the information you have, it is useful information that correlates with (future) reality that matters to you, and so it's prudent to use it. Appearing "nonjudgmental" by throwing away information and judgment is simply stupid, and bound to get you bitten at least in the metaphorical way.

but nobody in this entire thread was saying that, and my contribution was an additional attempt to further point out why the other sibling responses were missing this

At this point I'm totally content in talking past each other on this topic as I honestly don't know why its not clear that the subset which is undetectable by mere observation is statistically significant.

I’d argue we learn everything based on appearance. You have to do more than just observe things superficially and take context into account, but for something to be measurable it has to have some sort of outward appearance, whether that be direct, as in something you can see with your naked eye, or indirect, as in something we need to measure through some other instrument.

There are also cakes that look like cars, model sets which have miniature mountains, on and on. But in typical nature, it works well enough to breed and see what comes next.

1. If you look at the consensus in the field of psychology, human beings are probably not that good at making the kind of assessment you're talking about with any degree of accuracy. For instance, once you form a belief that there is a difference in competence between people of different races, you will become heavily biased to attend to examples which confirm your belief, and you will be biased to disregard examples which contradict your belief. Humans are just not perfect bayesian reasoning machines.

2. In the social contract of western, liberal society, we have broadly agreed that people should not be judged by their immutable characteristics. So in a sense, yes you are not allowed to "pattern match by race", according to the standing rules of society.

3. This rule helps protect us from the dangers of racial thinking. For instance, a belief about the relative merit or competence of one race can easily jump the gap from being descriptive to prescriptive when one goes around treating people by different standards based on their group identity. There is so much variance within every demographic group that even if there were some statistically significant difference between groups, you would be doing a massive injustice to so many of the individuals in that group to judge them by the "representative member".

I would say ruthlessly coerced.

https://en.wikipedia.org/wiki/Race_and_intelligence

Yes, obviously these statistics would not definitely tell anyone if there is any difference in actual genetic races, or if the factors are socioeconomic, but for someone looking for job candidates it would actually make no difference, since IQ is not generally easily changed or trained in a developed adult.

Of course, they could also just be using their own personal experience and claim that they are basing their beliefs on it.

I'm already slightly inclined to suspend my disbelief, because I know a little bit about developmental psychology that sort of corroborates what you're claiming about the ageism thing, however, if you want to start discriminating based on race, you have to be ready to bear a massive burden of proof when you make racist claims. Would you be so blindly accepting if I threatened your life, liberty or property for no reason other than the color of your skin? Wouldn't you want conclusive evidence, quality controlled and beyond a reasonable doubt?

>however, if you want to start discriminating based on race, you have to be ready to bear a massive burden of proof when you make racist claims. Would you be so blindly accepting if I threatened your life, liberty or property for no reason other than the color of your skin? Wouldn't you want conclusive evidence, quality controlled and beyond a reasonable doubt?

Would you not say the same about age?

Because when you ring up an insurance company and ask why (as a 20-year old) your insurance is higher than that of a 25 year old(with also zero driving experience) you will be told that it's because of your age. Unless they are lying to you of course, which is hard to prove or disprove.

>> however, if you want to start discriminating based on race

Wait, what? That took a weird turn?

Well, you turned first. Here we are all talking about racism, and then you claim age inversely affects driver risk. In context, I thought you were somehow trying to convince me that circular reasoning and appeals to authority ought to dispel any doubts I have about the validity of racism, but it sounds like you were trying to say something else, and I could be doing a better job of understanding what you are trying to say.

>unless they are lying

They may just be ignorant. Maybe someone else lied to them and they are just parroting the lie. Remember, the people who price insurance exist in a world where polygraphs are admissible evidence in a U.S. court of law; misconceptions are everywhere and I think we must be cautious, lest our generalizations be hasty.

Also see: invoking emergency military powers during peacetime (as reaction to overblown external threats) to suppress political dissent.

Yes, there are white separatist paramilitaries operating within the United States who have cropped up in reaction to affirmative action and social justice, but it won't blow up the same way the black panthers did. It's apples and oranges, because today's U.S. white separatist paramilitary groups are not targets of the U.S. government. The FBI isn't assassinating members of the Aryan Brotherhood nor are politicians passing gun control laws to curb the flow of weapons into their territories. The U.S. government is not actively persecuting white supremacists. There is no COINTELPRO for white supremacists.

There is no escalation here, no conflict. Nazis get a free lunch in the U.S.

It this were the case, babies would be racist by default, but they are not. Racism is taught.

"And when one user uploaded an image of the Democratic presidential candidates Andrew Yang and Joe Biden, Yang who is Asian American, was tagged as “Buddhist” (he is not) while Biden was simply labeled as “grinner.”"

They are not removing images that categorize blacks as blacks. They are removing images that are incorrect.

Racism starts out of preferential treatment of some people. Most racist people have a « root event », where the other party didn’t get condemned. It may even have happened several times, with various consequences (rape, molestation, repeat racket, etc).

Then they proceeded to denounce the rape/molestation/racket to the police. The police doesn’t act because they suspect you are racist. Which has the opposite effect than desired: It doesn’t condemn the criminal, and puts the burden on the victim.

Then they seek security in their lives. Therefore, if the probability is high to experience rape, molestation, racket or crime, enough that the racist has been confronted to it in his life, then the first best approximation of judging whether someone might be a criminal is whether he’s free or in jail. That’s in well-functioning societies. In non-functioning societies where criminals are not in jail, the second approximation to protect yourself is secondary indicators, inferred from grossly racist statistics. Here racism is born.

It also explains why racist people often still have various friends of the group they are supposed to mistrust. It’s because they were able to assess their probity and trustworthiness in some opportunity. That’s why mixing people by compulsory rules works. But mixing people is a poor palliative. It doesn’t solve the underlying problem if a non-functioning society, so it doesn’t make the people less racist, despite giving the appearance that people work together.

Racism is often used with regret by those who exert it. But it is the second best approximation to seek security. Racism is the result of a non-functioning society which is caused by being more lenient on criminals of chosen category. I’m pretty sure it is possible to engineer racism by letting a made-up group get away with crimes.

But you can find the same discourse in most famous racists of the world, at least currently living.

If I have to cite a source, Tommy Robinson (here comes the downvotes) constantly talks (and breaks) about the omerta surrounding the trials he reported on, which made that the rape gangs could proceed for so long with so few hindrances. Go through the list of people recently suppressed from Youtube and you’ll find the same (unadressed) logic.

Concerning root events, they are often private, so you have to know the person personally. But go ask around you when people became racist, they’ll often tell you one specific event. It’s interesting to go interviewing an enemy.