Unless you have thousands of fired and promoted employees, it may easily end up "Sorry, most of our promoted employees are Indians, because they were our founder's close friends and joined earlier, and that one guy fired for flirting with a customer was the only one from UIUC. Your name doesn't sound sufficiently Indian and you graduated from UIUC. Bye."

Worse, the person who looks at the rejection decision may have no idea that it boils down to this.

word2vec[0]:

      computer programmer
    - man
    + woman
    ---------------------
    = homemaker

Basilica?

[0] - https://arxiv.org/pdf/1607.06520.pdf

Note that some of this research, especially early, overstated the 'bias' here because they didn't realize that the default 'analogy' routines specifically rule-out returning any word that was also in the prompt words. So, even if closest word-vector after the `man->woman` translation was the same role (as is often the case), you wouldn't see it in the answer.

Further, they cherry-picked the most-potentially-offensive examples, in some cases dependent on the increased 'fuzziness' of more-outlier tokens (like `computer_programmer`).

You can test analogies against the popular GoogleNews word-vector set here – http://bionlp-www.utu.fi/wv_demo/ – but it has this same repeated-word-suppression.

So yes, when you try "man : computer_programmer :: woman : _?_" you indeed get back `homemaker` as #1 (and `programmer` a bit further down, and `computer_programmer` nowhere, since it's filtered, thus unclear where it would have ranked).

But if you use the word `programmer` (which I believe is more frequent in the corpus than the `computer_programmer` bigram, and thus a stronger vector), you get back words closely-related to 'programmer' as the top-3, and 23 other related words before any strongly-woman-gendered professions (`costume_designer` and `seamstress`).

You can try lots of other roles you might have expected to be somewhat gendered in the corpus – `firefighter`, `architect`, `mechanical_engineer`, `lawyer`, `doctor` – but continue to get back mostly ungendered analogy-solutions above gendered ones.

So: while word-vectors can encode such stereotypes, some of the headline examples are not representative.

One thing I’ve been tempted to research but never had time for myself: can one use that aspect of wording embeddings to automatically detect and quantify prejudice?

For example, if you trained only on the corpus of circia 1950 newspapers, would «“man” - “homosexual” ~= “pervert”» or something similar? I remember from my teenage years (as late as the 90s!) that some UK politicians spoke as if they thought like that.

I also wonder what biases it could reveal in me which I am currently unaware of… and how hard it may be to accept the error exists or to improve myself once I do. There’s no way I’m flawless, after all.

> For example, if you trained only on the corpus of circia 1950 newspapers, would «“man” - “homosexual” ~= “pervert”» or something similar?

If it did, what conclusion would you be able to draw?

As far as I know, there's no theoretical justification for thinking that word vectors are guaranteed to capture meaningful semantic content. Empirically, sometimes they do; other times, the relationships are noise or garbage.

I am wholeheartedly in favor of trying to examine one's own biases, but you shouldn't trust an ad-hoc algorithm to be the arbiter of what those biases are.

I think there's a further problem that there's never been a shortage of evidence, about things like this. The point is, prejudice and discrimination are not evidence-based in the first place. People who support existing unjust structures are generally strongly motivated to turn a blind eye. Even people who don't support them are - it's simply far easier and more socially advantageous to stop worrying and love the bomb.

I think this is a large part of what goes on in the digital humanities - to varying degrees of success. The problem, as usual, is not that there isn't an abundance of evidence. It's simply that nobody reads sociology papers except sociologists.

In this formulation wouldn't Basilica reflect the existing biases of the organization?

    resumes of candidates
  - resumes of employees you fired
  + resumes of employees you promoted
  ---------------------------------------
  = resumes of candidates you should hire

It's a lot of hard work to reduce bias in promotions and terminations.

Basilica might reinforce that hard work when evaluating candidates.

Or you could use the techniques described in your citation to allow Basilica to help de-bias the hiring process.

The second one being (I assume this is your point) merely a way to copy of all your existing biases, but not be able to see it.

eg If you fire all the black people and don't promote women, guess what resumes Artificial Intelligence will send you

I don’t think this approach will give you a good signal.

For the most part - People don’t get fired due to their skills. They get fired for lacking in execution or behavior. Someone screws up production deployment or makes lewd comments on another coworker. This is hard to come across in a resume.