I agree that there's too much blame-the-algorithm sentiment here, but if I had to guess, I'd think the author would argue that these racist biases should and could have been caught before going into production -- i.e. it doesn't have to be an inevitable cycle of ad hoc public shaming and repentance. These biases could be identified just as other SERP problems are identified before each algorithm update.

Just like the HP "racist" webcams [0] might have been caught had the CV trainers been more aware of what is/isn't in the training data, and/or if HP had a few more testers of darker complexion.

And perhaps Facebook's "Year in Review" rare tendency to be "cruel" [1] would have been mitigated with a broader group of devs and testers.

[0] http://www.cnn.com/2009/TECH/12/22/hp.webcams/index.html

[1] https://meyerweb.com/eric/thoughts/2014/12/24/inadvertent-al...

> I'd think the author would argue that these racist biases should and could have been caught before going into production

I think people who believe this significantly underestimate the difficulty of solving these problems.

I think that's likely. But the people tasked to solve these problems aren't always given the right priorities. Detecting/classifying the faces of dark-skinned humans is inextricably more complicated than it is with light-skinned humans. It requires significantly more training data, more computational power, and more testing time to reach the same accuracy as a light-skin-only camera and classifier.

We would deride Uber's self-driving vehicles if it turns out their systems can't tell the difference between harmless (plastic bags) and dangerous (jaywalkers, wildlife) objects on the road. Likewise, we can judge a company for releasing a general consumer webcam that fails to fully function for 10-15% of the American population.

>Likewise, we can judge a company for releasing a general consumer webcam that fails to fully function for 10-15% of the American population.

But then you get into an even deeper question 'are minorities/blacks 10% of said consumers of products?'. Poverty, for example, could mean that the product doesn't work for 5% of the buyers, which then begs the question 'how much effort do you put in fixing the problem for a small percentage of the buyers?', especially when the amount of effort is going to be very large/time consuming/expensive. If you are a person looking for racism, you will judge google as racist. If you are looking at it from a position of a business attempting to make a profit on a product, you will not see it as racist.

> But then you get into an even deeper question 'are minorities/blacks 10% of said consumers of products?'.

Maybe run that question by legal first! Protected classes exist for a good reason (IMO). What percentage of a restaurant's business is from people in wheelchairs, yet the law mandates ramp access.

This is very true. I suppose a mirror isn't quite the right analogy for Google, either. It's like film, a snapshot of what it thinks life is like. And for film, there are lots of technical problems for filming black actors, the likes of which had been solved 60 years prior for white actors[0]. It's very possible that the algorithm, captures a version of reality that doesn't show people of color in a good light.

And yes, these problems should be solved, and Google should use its editorial control to do the right thing. I believe it has done so, though perhaps not proactively enough. But these are hard problems, ones that are still not solved in other industries. Web search is still a young industry.

[0]: https://mic.com/articles/184244/keeping-insecure-lit-hbo-cin...

I'd argue that most of the 'algorithms are evil' crowd don't even understand the problem to begin with since many of them do not come from a technical background.

How much does the average driver have to understand about automotive and mechanical engineering before they can form a reasonable criticism about defective/fatal design?

Physical products are much easier to intuit your way around than software. For cars, effectively everyone knows what the engine, fuel tank, tires, and drive selection lever are (though they may call the last one a "PRNDL"). Even if they don't know exactly how an alternator works, they understand that they need to drive their car around after they jump it to recharge the battery.

By comparison, I think most people don't even know the basic architecture of a computer, let alone the applications that run on it. Most people don't have an understanding of the network stack - I consider people ahead of the curve if they know the difference between LAN and the wider internet. Some people even think it's possible to download more RAM...

Driving follows physics mechanics, is a skill and is much more involved than search. Search is designed to be a black box for most people.

But ignoring that, no, I still would not like to see people professionally criticizing processes and products that they do not understand. Some technical knowledge is a must for people who do professional talks or presentations, write books and research papers and teach on the topic.

For algorithms and ML, I would expect at the very least a bachelors in math/CS or the equivalent in work/personal project experience.

criticism follows understanding, so the only honest answer is “quite a lot”

i realize this isn’t widely appreciated or accepted in the age of critical theory, when social desirability leads people to complain about everything in the guise of “fighting oppression,” but it’s a simple real-world fact that no one solves problems from a position of ignorance

I don't understand. Can't these and other sample search terms be added to the regression tests before new versions of the algorithm are deployed? Then you'd likely see the errors in time to correct them.

I think it's more that we think that they're not even bothering to test those things, because they're not thinking about it.

How would you test something like this, if not aggregating user reports? Attempting to preemptively think of every possible thing that could be construed as politically incorrect is a Sisyphean task. It's a lot like the "H8GAYS" airline confirmation code that made headlines [1]. Delta probably blacklisted all profane words, but there are still combinations that can be perceived as offensive. Similarly, I once worked at a summer camp teens that generated users' passwords by picking two words and two numbers (all profanity was blacklisted as were 69, 88, 13, and I think a few other numbers). That didn't stop a camp attendee's parents from getting called in because their daughter was assigned the password "bloodyunhappy12" (the concern was that "bloody unhappy" referred to menstruation).

[1] http://newsfeed.time.com/2013/12/17/delta-airlines-is-very-s...

> I'd think the author would argue that these racist biases should and could have been caught before going into production -- i.e. it doesn't have to be an inevitable cycle of ad hoc public shaming and repentance.

They could have caught more cases before going to production - that's a truism. But when X number of cases get "fixed", there are other N*X cases/other communities/other minorities that won't, and there will be ad hoc public shaming that will caught enough attention to become relevant, and so on. Because the number of things that offend at least somebody is infinite.

My point is that there is no absolute way to tell what's offensive and what isn't - it's all subjective. There is no perfect way to fix them all - not even in theory, because there are conflicting priorities - and public outrage serves as an imperfect way to prioritize the most inter-subjectively blatant. Are there other ways? Sure, but they won't stop the periodic public outrage.

You’re missing the point, which is that you’re dealing with a coupled system (in the physics sense). Google reflects our racism, but exposure to that racism on the first page of results makes us more racist. Google concentrates it and reflects it back to us.

(Google does this with everything else as well. What do people think of the new MacBook Pro? Searching for reviews will immediately spit back a concentrated and distilled version the first mover opinions, which will itself drive peoples’ opinions.)

What's more sinister--Google accurately reflecting the gist of the content and information available on the web, or Google carefully scrubbing it to better reflect a chosen set of cultural sensitivities?

> Google accurately reflecting the gist of the content and information available on the web

I wouldn't assume that Google's search results reflect anything but an attempt to bring you back by showing you results you find valuable, and to monetize your visit.

> scrubbing it to better reflect a chosen set of cultural sensitivities?

Google (and any other mass market vendor) removes a lot of things that people don't want to see, including scams, X-rated content, brutality, etc. People searching for information on their dog don't want to see bestiality. These aren't arbitrarily chosen preferences or mere "sensitivities", but social norms which in many cases have a strong moral foundation.

"Social norms" can be either descriptive or prescriptive. Descriptive norms are norms that reflect society as it is, and Google doesn't need to do anything about them, because the society that produces millions of web pages with cute pictures of puppies or helpful information about what not to feed your dog for every single web page with dog porn will result in a SERP that is overwhelmingly weighted towards that information.

Conversely--haven't these same people been telling us for years that we live in a racist society? So why are they surprised that a search engine of a racist society's web pages returns racist results?

I'm surprised myself that someone on HN is really claiming that Google doesn't edit search results for all sorts of reasons, including for purposes of social norms, of any sort, and social responsibility. In fact, every person and organization does that. I doubt there are any social interactions where people don't behave according to a set of rules.

> why are they surprised

Nobody said they are surprised; that's not relevant. They are trying to solve a problem, just like people who are working to resolve the problems of cancer, gun violence, and bugs in Firefox.

I'm not saying they don't edit search results (clearly they do); I'm saying that it's worth reexamining the expectation that they should edit search results for the sake of political agendas.

You raise an interesting point, but the comment was about the article itself and the thesis in the article.

FWIW, I read the article as addressing the issue in general. The headline and lead-in are Google-specific (as well as many of the examples), which is unfortunate in the sense that it's arguably click-baity and leads to arguments like this, that the thesis is wrong because it's not just Google. I understand why it's done, as providing a familiar, well-understood context helps people better understand what the author is getting at.

Examples of how the author is approaching the topic more generally are found in headings such as "Content and Creators" and phrases like "Information monopolies such as Google". The author's book is likewise entitled "Algorithms of Oppression: How Search Engines Reinforce Racism", not "How Google Reinforces Racism".

I agree that results in Bing and Duck Duck Go, for example, currently show a lot of the flaws describe in the article. But the point (as much as the headline may lead you to believe) isn't that it's only Google.

rayiner wasn't just bringing up Google vs other companies, though.

rayiner was saying that there is a "coupled system" and Google just reflects external racism, and then Google's search results amplify racism. The article did not mention anything about those interesting points, which might be worthy of bringing up and discussing, but it seems very unfair to accuse the top comment of "not getting the point" about them, when they are rayiner's insights that were only now presented.

Your second paragraph explains why Google, the search tool, has a striking history of bias against black women, by stating "search is a reflection of [..] how terrible we are". Your third paragraph explains that Google, the company, spends time and money on correcting bias.

I don't think that's news to the author, or a refutation. She says tech companies produce tools in environments of racism/sexism, without proper safeguards, and the result is damaging to people of color and women. She's saying Google can and should consider this stuff in advance, and take responsibility for bias in their products (even when it's the reflection of ugliness in third parties as you said).

So, I don't see your perspective as being in conflict with her. Except inasmuch as you excuse Google for reflecting our biases. Do you think they should feel OK if their next AI winds up with our biases? If so, why? It doesn't feel like you've explained that.

I didn't see that comment as excusing anything.

I think we all agree Google should do as much as possible to improve results. But at the same time, if Google gets it right 99 times out of 100, the 100th is what will get an article like this written on it. That doesn't excuse that, it explains why it happens.

> Your second paragraph explains why Google, the search tool, has a striking history of bias against black women

The article says "Google has a striking history of racism", which you interpret as "Google results have included things which are strikingly racist." That's one interpretation, but it seems equally valid to interpret "Google has a striking history of racism" with "Google has been very racist" (after all, if I said, "Bob has a striking history of racism" you'd be right to assume I think Bob is a racist).

I think some of the controversy around this article stems from these interpretations, one of which is a far worse accusation than the other.

> if Google gets it right 99 times out of 100, the 100th is what will get an article like this written on it

Is there any substantive basis for this claim? My impression is that Google gets away with a lot.

By having a mirror, we can see and correct our flaws.

I used to be (well, still am, a bit) worried that the increasing automation of our social structures (credit scores, admission processes, etc) will just encode these biases in ways that cannot be altered.

But then saw how much of NIPS was about "fairness".

If these biases get encoded, they are measurable, examinable, detectable - and can then be accounted for in ways (presumably) much more effective than existing patterns.

So now, I'm not worried about the technology, although as always, I worry about the application of it.

Is the (assuming it's true, but it seems plausible) situation that the majority of users searching for "black girls" actually do want to find sexual content instead of simply smiling faces a flaw that needs correcting? Is their intent unfair in some way, are their sexual preferences unacceptable?

If I don't like what I see in my mirror, will putting makeup on the mirror improve my face? Will removing naughty sites from the search results somehow change the fact that "black girls" (or "asian girls", or "czech girls", etc) is not usually used as an innocent search term?

So, I'd dance around the issue.

Google's search algorithm is "fair", but society is not: It is not "fair" that these search terms are over sexualized.

This is, of course, given some particular morality and ethos; your may not find society unfair in this way. (I do)

My point is that now it is visible and you can make more informed choices.

If you don't like what you see in mirror, why is makeup the "fix" of choice? Alter diets, exercise patterns; sun exposure, fitness levels, stress levels...

> Will removing naughty sites from the search results

Potentially. We're feedback loops as well; if these terms result in what someone is after, they will use those terms: if they don't, they won't. Eventually, I would expect people to stop trying to search with them; to then stop communicating with them (as meaning that); to then stop thinking with them (as meaning that).

My point was that it's like applying makeup to the mirror instead on your face - what you propose is not like improving your fitness, it's like making a curved mirror that makes you appear slim and flatters you, hides the problem and doesn't change anything about the underlying issues.

After your proposed changes it is no longer visible, and we can no longer make informed choices - people searching for "black girls" still want the same thing, but they're getting a worse service (results that don't fit their intent) just to hide them and their preferences from other parts of the society. There's also a feedback loop in the opposite direction, if there are real problems that simply are hidden, they're not going to get fixed.

On the other hand, it's worth examining the actual problem - it's worth noting that people who do want to find sexual pictures of black girls are just as valid users as black girls, and any fair solution must keep both of their preferences in mind; it would be unfair to simply damage the service for one group to appease the another. IMHO a fair solution would be based on data about what the people who use that query actually want (in some equivalence to picking equal opportunities in the "equal opportunities vs equal outcomes" tradeoff dilemma) - if some people are using the term in their daily search and some are not, then the interests of these frequent users could reasonably be considered more important. Aggressive personalization (different users having wildly different search results for the same term because of the differences in the assumed intent) is another way to do that, but it brings a bunch of other problems.

To be clear, I haven't said anything about my proposed changes. I'm parroting what I understand to be the changes Google actually applied.

That said...

You're missing that some problems are, actually, entirely perceptive. You're right that changing the mirror to hide problems isn't addressing the underlying issue, but sometimes treatment of symptoms is the only available (or possible the most effective) option. Like everything else, it's one tool that can be applied poorly or effectively.

> it would be unfair to simply damage the service for one group to appease the another

This is ineffective as a stand-alone principle. The failure mode is described as "your freedom ends where mine begins" and is clearly visible in the exaggerated (and somewhat contrived) example of "damaging services for murdering to appease those who want to remain un-murdered is good".

Like all principles, this has both extreme-case failure modes (services for murderers) but also fuzzy-case failure modes. If the term for my identity is sexualized, when that identity has nothing to do with sexuality... well, what's fair? Why should group X define the term instead of group Y?

So, for cases where the problem is primarily perceptive (or the symptoms are primarily perceptive), seems reasonable to me to warp the mirror. It's like using glasses: you're already looking through a warped lens, so use a correspondingly warped lens to correct.

My actual point is that systems that incidentally encode biases then let you better examine those biases, and, depending on the situation, do stuff to address them.

What you're proposing is a hack, and one with a lot of negative side effects.

First of all, to go with the obvious, you better hope that what Google deems the direction they'd like to alter reality is aligned with your own, because I'm quite sure that this will mostly be untrue.

Secondly, you're attempting to patch a problem at much too high of a level; making people less racist by altering what they're able to ingest, particularly by altering unbiased reality, is a bandaid that is destined to fail hard. What if, say, we wanted to change the negative stereotype of black Americans engaging in criminal behavior at substantially higher rates, so we demanded that Google remove access to racially distributed crime statistics. Many countries already do this, such as Sweden. We will have lost a primary tool with which to assess the nature of our reality, contextualize it, and strategize about it effectively.

I'm more interested in a free society bettering reality than I am with hoping that Google's ideal version of reality is aligned with my own... particularly because I would still consider it to be a horrible solution even if it was.

Making people less racist by altering what they're ingesting is one of the most effective ways to do it.

The "reality" in this case is not unbiased, because it's not the reality of "cars are hard" but the reality of "red means stop".

> access to racially distributed crime statistics

That does sound like something that would backfire or fail. Seems like you hit on the delineating factor: You search for crime statistics to contextualize and understand your reality. Is that why people search for porn?

Google can amplify the result though. Just like a population that votes 51% Republican and 49% Democrat can have a 100% Republican legislature if divided into equally representative districts, Google can amplify society’s latent racism by making a model that is more racist than the training data.

This is a good point, but I also think that any search engine is inherently an amplifier, so it's simplistic to say they should just fix it somehow.

Right. The title ought to be something like "English-Speaking Cultures Have a Striking History of Bias Against Black Girls". Or "... Women of Color" if that's your preferred language. Google Search is arguably just a mirror.

And ideally, that's what it should be. Otherwise, how do researchers know what's really there, and what Google makes up? But in fact, Google is far from that ideal. Maybe it was initially unbiased. But successive waves of fighting SEO, and responding to social pressure, have taken their toll. Not to mention the increasing focus on what's happening right now, forgetting the past, and showing searchers what they likely want to find.

> Google Search is arguably just a mirror. And ideally, that's what it should be.

I disagree this is the ideal. The first implementation of PageRank makes a very big assumption: that people who write webpages and control websites can be trusted to link to worthwhile content. If webmasters and web producers skew a certain way, then PageRank/BackRub will also skew that way.

Google original mission statement was "to organize the world's information and make it universally accessible and useful." Being a mirror of web content would only partially solve that mission, and that's assuming webmasters/producers are acting in good faith. Would it really be ideal for Google to be a "mirror" of link counts for "miserable failure" [0] and "Did the Holocaust Happen?" [1]

[0] https://en.wikipedia.org/wiki/Google_bomb

[1] https://searchengineland.com/google-holocaust-denial-site-go...

No, Google does not try to mirror content. It tries to provide the best results for users. Results the user wants to click on and are relevant. Searching for "professor style", nobody wants it to return a minority professor style. Even she doesn't, at scale. If she and others did, they'd be Googling "professor style", not clicking, then Googling "professor style for black women" and results would improve/personalize.

Ditto for "black girls": either your looking for porn or for stock photos. What other intentions do you have with such a query? (Besides making a point?)

Google changing results for Holocaust "denial" is likewise wrong. Someone searching for such things probably wants to find all such sites and not just a PC version of history repeating what they've already heard. Otherwise they wouldn't need to be searching in the first place. Same for someone searching "Is the moon real" or "is the earth flat".

In a great market, Google might start "cleaning" up their results to the point of annoying people, who would then switch. Unfortunately no one comes close to Google search, let alone overall lock-in. So we're stuck with whatever Google decides reflects their brand, including fixing up well-publicized queries for people that want to be offended and complain about stuff.

Edit: I search for "best basketball player" and all I see are black guys -- way over-represented. Why doesn't Google bump up Larry Bird to 4th place so there's some more diversity and fight the stereotype? Even worse, Yao Ming is nowhere to be found, what's that say?

Sorry, just to clarify, I (and I believe the GP did as well) meant "mirror" in terms of reflecting reality and the state of things -- not "mirror" as it's commonly thought of in content/data online distribution.

I think you misread the Holocaust denial situation. The reason why "Did the Holocaust happen?" was so problematic was because that literal string of text is something that you'd never find on a mainstream site, such as the NYT or the Holocaust museum. And yet sites like StormFront would get bonus SERP for using that literal text in a page's title, url slug, and <h1>. StormFront benefited from a blind spot in Google's heuristics that had little to do with whether or not StormFront was actually a legitimate researcher into the question of the Holocaust's historical reality.

Same problem with Google Bombs. A law student with a blog was able to get enough people to link "waffle" to 2004 presidential candidate Sen. John Kerry [0]. Presumably, the average person googling "waffle" wants the food item, not info about Sen. Kerry. But the reality was that the metrics Google had long used to measure quality and credibility were all saying "waffle => Kerry".

Staying true to the algorithm assumes that the algorithm (and its inputs) were pure and complete to begin with. This is not necessarily the case.

[0] https://en.wikipedia.org/wiki/Political_Google_bombs_in_the_...

> Sorry, just to clarify, I (and I believe the GP did as well) meant "mirror" in terms of reflecting reality and the state of things -- not "mirror" as it's commonly thought of in content/data online distribution.

Yes, I did. But I don't want to see the parasitic SEO-driven copypasta linkfarms. Or perhaps have them, but including such flags.

> Ditto for "black girls": either your looking for porn or for stock photos. What other intentions do you have with such a query?

Consider the possibility that the user is a black girl, or her parent. Generally, I'm careful not to assume I can anticipate the perspectives of everyone else.

Do white boys ever enter the query "white boys" into a search engine? I assume some HN users could speak to this...

To answer your immediate question, "white boys" is not uncommon in popular usage. People might look it up in the process of looking up a lyric or obscure title/name for a song/group.

The phrase "white boys" in general seems to be mostly desexualized. You can call someone a "white boy", either as a mild insult (like describing something as "vanilla") or a term of endearment. I think that's predicated from most people (in the U.S.) realizing that white boys/men are the status quo.

I'll ignore race and just say that I think the word "girls" in general has far more of a sexual/gendered connotation in U.S. usage. "Boy/Man" is considerably more generic and all-purpose. I use "Boy/Man/Guy" in casual conversation regardless of audience gender, e.g. e.g. "Boy, that was fucked up!" or "Man oh man that's got to hurt". I can't think of a single time in normal conversation where I would ever say "Girl" or "Lady" without referring to a female.

I'm aware of how the term is used, but your comments seem more like a tangent than an answer.

My question was whether there is any connection between being a white boy and Googling the search term "white boys".

It was prompted by the parent post suggesting that not seeing a connection between being a black girl and Googling "black girls" might be kind of a blind spot.

I think the implication was pretty clear that if you don't imagine someone Googling the search term "black girls", it's because they occupy the position of the "other" in your mind. However, I gave a reason to doubt it - contemplate whether and how often white boys Google "white boys".

I note that somehow, I already knew the connotations of the term "white boy" that you mentioned without Googling it. So you are not bringing anything to the point I was making.

I am not a laissez faire fanatic about much of anything, including Google search results - I just think that when people criticize such results, the search terms (and the subsequent reasoning) often seem contrived.

Ok so why is she searching for "black girls"? I don't search for "Canadian guy". Outside of wanting a generic/stereotype photo, what're the reasons?

Same for "professor style" and expecting to get something other than that stereotypical brown sweater with elbow patches look.

Well, I'd like some search service that shows me what's accurately out there. If there's content out there about somebody being a "miserable failure" or Holocaust denial, I want to see it. I mean, what good is a search service that doesn't show what's there?

If I don't want to see whatever results are useless or offensive, I'll do my own filtering.

But then, I grew up on searching the scientific literature using Science Citation Index. And other stuff in LexisNexis. You don't expect stuff to be missing because it isn't cited much, hasn't been replicated, or whatever. You can filter by how well stuff has been cited, of course.

> I'd like some search service that shows me what's accurately out there

What does that mean, to accurately depict what's "out there"? That could be as nonsensical and unproductive as completely killing the spam filter in your email. Do you feel that that GMail and other modern-spam-detecting solutions have given you an inaccurate view of the "real" mail you are getting?

By the late 2000s, content farms such as Demand Media were churning out clickbait that followed the best practices with SEO. Demand Media [0] consistently beat out the NYT on SERP, thanks in part to a store of 1 million domain names used to serve up ads and links for topics ("howtomakeasandwich.com" would like to junk articles about sandwiches). By 2010, DM had 105 million unique visitors a month [0] and reached a market cap of $2B. The NYT, by comparison, had about 32M unique monthly visitors [1] -- across all of its properties (BostonGlobe.com, etc), and its market cap in 2011 was about $1.5B [1].

Demand Media beat the living shit out of NYT and every other quality media site by playing to the the known heuristics of Google's SERP. Every other media company had the opportunity and resources to play spamlord. Apparently they didn't, and by 2011 no other media company was positioned to create content (of any quality) with the scalable efficiency of Demand Media.

Google could have just accepted that "might makes right", but that is not an objective/neutral decision, nevermind a rational one.

[0] http://variety.com/2013/biz/news/epic-fail-the-rise-and-fall...

[1] https://www.theatlantic.com/business/archive/2011/12/it-cost...

I don't like spam filters. Because I've been burned too many times, missing important messages that got flagged as spam. So I end up having to look through all the spam anyway.

I did hate the Demand Media bullshit, however. But as I've said, I'd rather that Google flagged that stuff, and let me filter it out. I mean, maybe I'm researching an article about Demand Media. How could I do it if Google is suppressing it? Could Google perhaps have an "expert" option? Analogous to "family friendly" or whatever it's called.

The Science Citation Index and LexisNexis already do most of the filtering for you by not including absolute unmitigated garbage that happens to be perfectly optimised for your search keywords. You might find the odd David Irving article in there, but you haven't got Stormfront trying to citation-bomb all holocaust related terms for the lolz, never mind commercial blogspam. And Google also works on the assumption most users won't do their own filtering.

I can't imagine that Science Citation Index excludes stuff from the literature. But it's been a while since I used it. Perhaps they drop "absolute unmitigated garbage". Or that they neglect to index obscure journals.

Where are the filters that show which search demographics contributed to a result being relevant? I could use that to filter out SEO's, googlebombs, racist results, etc., if I wanted.

Perhaps, but I doubt it.

As one anecdote, http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a... reports facial recognition algorithm performance on darker-skinned women up to 30% lower than lighter-skinned men. The preliminary results were reported to Google, IBM, and Facebook. IBM was able to tune its algorithm achieve comparable scoring across all skin hues/genders within a week (according to my notes from the talk). It could be that search on that scale is much harder than facial recognition, and its also that a quick fix would introduce other errors, I'm sure it's an NP-complete problem, yada yada.

But Google and racialized search is not a new issue https://dataprivacylab.org/projects/onlineads/1071-1.pdf.

Yes the society is racist, but then the U.S. enacted the 13th, 14th, 15th and 19th amendments to address extreme racism and gender discrimination. That is, citizens -- responsible engineers and data scientists among them -- can take it as their obligation to confront systemic racism. Yes, Klansmen are racist, Nazis are racist, but we don't have to accept genocide -- we don't even have to accept biased, discriminatory and derogatory internet services either.

I have to agree with you. You can go to any country with a fetish of some sort and have "surprising results" for innocent searches. Go to google.co.jp and search for "Russian Women" [ロシアの女性] Is that a racist algorithm?

> Google likely spends more money on editorial control for issues of racism than for any other thing that doesn't make money.

Is there evidence to this effect? There are a lot of things Google spends money on.

Good point; it's loose speculation, if even that, and I'm surprised to see it get so much attention.

I'd add that it does make money. Making your customer experience pleasant is an essential money-maker. If people are loudly saying racist things in your restaurant, the financially essential decision is to exercise editorial control before you lose your other customers.