News for hackers. Stuff that matters.

Cloud Vision API will not return gendered labels such as 'man' and 'woman'

I just received an email from Google Cloud Platform:

Hello Google Cloud Vision API customer,

We are writing to let you know that starting February 19, 2020, the Cloud Vision API will no longer return gendered labels such as 'man' and 'woman' that describe persons in an image when using the ‘LABEL_DETECTION’ feature.

What do I need to know?

As you know, the Cloud Vision API can perform feature detection on a local image file for the purpose of identifying persons by sending the contents of the image file through ‘LABEL_DETECTION’.

Currently, when you request the API to annotate an image with labels, if you use this feature on images with people, it may return labels describing them in an image with gendered terms, like ‘man’ or 'woman’.

Given that a person’s gender cannot be inferred by appearance, we have decided to remove these labels in order to align with the Artificial Intelligence Principles at Google, specifically Principle #2: Avoid creating or reinforcing unfair bias.

https://ai.google/principles/

( 39 comments | 47 points | on HN )
Re:Cloud Vision API will not return gendered labels such as ... by 0x1221 on Thursday February 20, 2020 @06:58AM (#22374416)
> Given that a person’s gender cannot be inferred by appearance, we have decided to remove these labels in order to align with the Artificial Intelligence Principles at Google, specifically Principle #2: Avoid creating or reinforcing unfair bias.

Isn't this just outright wrong? A person's gender can be inferred by appearance, and with extremely high accuracy at that.

Re:Cloud Vision API will not return gendered labels such as ... by rl3 on Thursday February 20, 2020 @07:53AM (#22374702)
While I tend to agree with you from a technical standpoint, I can't help but think Google is actually doing the morally correct thing here.

If organizations are creating systems using that API, most will just roll with man/woman labeling. If those systems are important, a lot of trans and non-binary people will end up fucked over.

At the end of the day, if the deprecated functionality is still needed, just set a threshold on masculine vs feminine. It's a far more elegant approach, even if more work.

Re:Cloud Vision API will not return gendered labels such as ... by johnlorentzson on Thursday February 20, 2020 @08:07AM (#22374781)
Automatic gender detection should certainly not be used for data entry where the data is of any real consequence (even if we ignored trans/non-binary people), simply due to the fact that it can't be proven to be perfect.
Re:Cloud Vision API will not return gendered labels such as ... by muzani on Thursday February 20, 2020 @09:23AM (#22375240)
I think yeah, it's easier to use masculine/feminine. Something like long hair, soft skin, pink cheeks, more curves is feminine. Someone with more squares/edges (jaw, cheeks, shoulders) is more masculine.

Humans have trouble with this too, e.g. many crushed on Taylor Hanson.

Re:Cloud Vision API will not return gendered labels such as ... by brodouevencode on Thursday February 20, 2020 @11:09AM (#22376047)
I always get it confused: is it sex or gender that people can assign to themselves?
Re:Cloud Vision API will not return gendered labels such as ... by foldr on Thursday February 20, 2020 @01:02PM (#22377048)
Are you really confused about this, or just rhetorically confused? You can answer the question by googling "sex vs gender".
Re:Cloud Vision API will not return gendered labels such as ... by comsolo on Thursday February 20, 2020 @04:05AM (#22373743)
Another instance of SJW types causing problems in the coding world (although the vast majority of actual coders aren't on the SJW train).

The fact is gender can be determined by appearance in 99.5% of circumstances or more, which is probably higher than the overall accuracy of their ML labelling system.

The idea of "avoiding unfair bias" is highly vague and problematic since, theoretically nearly any statement or view beyond "particle X is at Y" type statements has a somewhat "unfair" bias reflecting the lens of the human experience, which has developed over millions of years of biology and thousands of years of culture.

Finally, seeing as the fact is that 99% of people would not be negatively affected by this feature remaining whatsoever, and the very slight negative impact on the other (intersex type) 1% or so is minimal at best, while the total impact of removing the feature is mildly annoying for many users, this seems a lot like irrational, hollow, virtue signalling.

Re:Cloud Vision API will not return gendered labels such as ... by foldr on Thursday February 20, 2020 @09:26AM (#22375256)
>the other (intersex type)

The biggest group likely to be impacted is trans people, not intersex people.

Re:Cloud Vision API will not return gendered labels such as ... by comsolo on Thursday February 20, 2020 @10:29AM (#22375700)
Still less than 1% of the population, and that number could dwindle if and when treatment sorts out the underlying problems with the mental wiring causing the gender dysmorphia, and the current trend dies out.
Re:Cloud Vision API will not return gendered labels such as ... by lordCarbonFiber on Thursday February 20, 2020 @10:52AM (#22375890)
Red heads are less than 2% of the population and yet if you built a classification system that failed to identify them as people that wouldn't be seen as acceptable either.

Trans people have been around forever, and aren't going away just to make image classification less complex. It's not a "trend", treatment options just happen to be more available and the internet makes communities more visible now.

Re:Cloud Vision API will not return gendered labels such as ... by ralfd on Thursday February 20, 2020 @11:39AM (#22376295)
> Red heads are less than 2% of the population and yet if you built a classification system that failed to identify them as people that wouldn't be seen as acceptable either.

So the solution would be to not identify _anyone_ as people? As a ginger I doubt that.

Re:Cloud Vision API will not return gendered labels such as ... by foldr on Thursday February 20, 2020 @01:00PM (#22377021)
The solution would be to accept that the classification algorithm was too error probe to be usable for non-toy applications.
Re:Cloud Vision API will not return gendered labels such as ... by ralfd on Thursday February 20, 2020 @01:33PM (#22377344)
We are going in circles. An error rate of 2% (heck! 20%!) would not make a classification unusable, that was what one of the grandfather posts was arguing.

https://adssettings.google.com/u/0/authenticated

Here Google can infer my gender and my age and personal interests just from my search history. I am sure this is not perfect either, it is still immensely useful for advertising.

Re:Cloud Vision API will not return gendered labels such as ... by lordCarbonFiber on Thursday February 20, 2020 @01:51PM (#22377529)
Not the best example, given google lets you toggle your ad segments when they get them wrong and isn't baisising that guess based on physical presentation; the ad segment "male" is just a comment on the things you search and click on.
Re:Cloud Vision API will not return gendered labels such as ... by natalyarostova on Thursday February 20, 2020 @06:49PM (#22379710)
Why are you imposing the error tolerances the user should have? Shouldn't that be their choice?
Re:Cloud Vision API will not return gendered labels such as ... by lordCarbonFiber on Friday February 21, 2020 @09:21AM (#22383257)
If someone wants to build and train their own model that assigns a binary gender based on pictures that's their own choice, and because of that freedom they can't force google to do it for them.
Re:Cloud Vision API will not return gendered labels such as ... by natalyarostova on Friday February 21, 2020 @12:40PM (#22385003)
Right, it's just that usually the concept with goods and services is that you're paying someone else to do it for you because you either can't or don't want to.
Re:Cloud Vision API will not return gendered labels such as ... by foldr on Friday February 21, 2020 @02:41AM (#22381558)
If I'm the author of the tool, I can impose whatever I want, no? It's not like people have a God-given right to have Google tag photos with genders for them.
Re:Cloud Vision API will not return gendered labels such as ... by EvgeniyZh on Friday February 21, 2020 @12:29PM (#22384893)
So the best argument to do is the fact that you have a legal right to?
Re:Cloud Vision API will not return gendered labels such as ... by foldr on Friday February 21, 2020 @06:04PM (#22387901)
The argument in this case is that the potential for misuse outweighs the potential benefits.
Re:Cloud Vision API will not return gendered labels such as ... by orangesquirrel on Friday February 21, 2020 @12:09AM (#22381068)
Depends on the application. For something like approximating demographics of retail store traffic, this kind of issue would probably be fine. I don’t think anybody expects these models to be right 100% of the time anyway.
Re:Cloud Vision API will not return gendered labels such as ... by hluska on Thursday February 20, 2020 @10:29AM (#22375701)
A more charitable interpretation would be that classifying gender is hard. It is so hard that humans have trouble classifying gender purely with sight. If humans have so much trouble, how can we teach machines?
Re:Cloud Vision API will not return gendered labels such as ... by esyir on Thursday February 20, 2020 @10:03PM (#22380629)
Classifying gender is easy. Humans do it correctly with near-perfect accuracy, typically failing only on edge cases. Furthermore, most cultures already bake in signaling of gender in various forms (Behavior, attire, etc).

Humans don't have much trouble, except against adversarial/outlier examples.

Re:Cloud Vision API will not return gendered labels such as ... by hluska on Saturday February 22, 2020 @07:16AM (#22390626)
Can you provide some citations for what you’ve written?
Re:Cloud Vision API will not return gendered labels such as ... by esyir on Saturday February 22, 2020 @10:41AM (#22391526)
First, please hold yourself to the same standard that you're holding me to. I see no citations to your (paraphrased) "identification of gender is exceedingly hard for humans". That goes heavily against most people's intuition and thus carries a higher burden of proof.

Now, 0.6% of Americans identify as transgender [http://williamsinstitute.law.ucla.edu/wp-content/uploads/How...]. That means using sex prediction alone, we can achieve a theoretical maximum of 99.4% accuracy here. This ignores the fact that transgenders often signal their gender via behavior or clothing, or attempt to obtain the physical characteristics of their assumed genders as well, both of which could further improve performance.

Re:Cloud Vision API will not return gendered labels such as ... by hluska on Saturday February 29, 2020 @11:11AM (#22451243)
I asked for your help. Why did you see fit to get rude? What the hell is wrong with society that people like you think this is okay? Toxic responses like this ruin tech because instead of teaching you attack.

Loads of research suggests 7 year olds are about 70% accurate at predicting gender based off of pictures. It takes a couple of decades to get that into ranges you’ve cited. That doesn’t sound easy. If it is, I’d love to read about it.

Why be toxic?

Re:Cloud Vision API will not return gendered labels such as ... by natalyarostova on Thursday February 20, 2020 @06:48PM (#22379702)
Should we not teach machines to do things, or attempt to do things, if they can't do a better job than humans?
Re:Cloud Vision API will not return gendered labels such as ... by he11ow on Thursday February 20, 2020 @03:39PM (#22378370)
Wow, the responses to this thread are such an excellent marker for skin colour. I don't know the Google stats, but as far as labeling black women as male, Amazon and IBM get it wrong 30% of the time. Microsoft errs in 21% of test cases. As for Google's rates, not all that long ago they were labeling black people as gorillas so... [0]

Put it another way, how impressed would your boss be if you were to say "Yo, I've nailed this. I've got a kick ass algorithm, check this out. Oh, yeah, it fucks up 30% of the time."

Bias in AI is bad, but what makes it much much worse is arrogant attitudes claiming it doesn't exist.

And you know what, that speaks to the idiocy around AI more than anything. All these debates on AGI are moot, because people are already so keen to believe they got it all figured out it doesn't even matter if the actual tech is any good at all.

[0] https://www.wired.com/story/photo-algorithms-id-white-men-fi...

Re:Cloud Vision API will not return gendered labels such as ... by raxxorrax on Friday February 21, 2020 @07:09AM (#22382528)
> Wow, the responses to this thread are such an excellent marker for skin colour.

As is pronouncing the effects of bias and prejudice. You can almost guarantee the person is white without using any AI.

I was or still am pretty critical of AI since the quantification of every property of people will probably not be very enjoyable for anyone. What changed my mind a bit is that there are people wanting to employ it to improve reality. That is far worse and I hadn't had that on my radar before.

Re:Cloud Vision API will not return gendered labels such as ... by fyp on Thursday February 20, 2020 @09:37AM (#22375334)
My google ads profile correctly identifies me as male: https://adssettings.google.com/

I imagine the high level business use cases for these machine learning APIs are similar. For example, analyze shopping mall camera footage to figure out demographics shopping there and coordinate the (socially accepted and segregated) men/women department stores accordingly.

This is such an profitable use case I cynically can't see why they would cripple their API like this, regardless of their ethical stance.

Re:Cloud Vision API will not return gendered labels such as ... by mateo1 on Thursday February 20, 2020 @12:17PM (#22376641)
That's just hilarious. We live in an era where the statistical differences between the appearance of men and women are better understood than ever before, thanks to artificial neural networks, and we can in fact pinpoint the exact combinations of low-level features that make a face look like a man or woman. And then you have people in such a deep denial of reality they pull off this type of joke. Sure, you cannot identify sex with absolute certainty only with a photo, but ignoring fringe cases you can guess with very high certainty.
Re:Cloud Vision API will not return gendered labels such as ... by emilfihlman on Thursday February 20, 2020 @12:22PM (#22376691)
It's funny how ads can be targeted at men or women, but everything else is just "nooooo, we can't do that!".

I wish the adbuyers would flex that money muscle a bit and put google in line again.

Re:Cloud Vision API will not return gendered labels such as ... by MidnightRaver on Thursday February 20, 2020 @03:49AM (#22373678)
I wonder if this is actually a marketing gimmick to invoke outrage and therefore clicks? The change certainly reduces the usefulness of the API by a fraction of a fraction of a percentage.
Re:Cloud Vision API will not return gendered labels such as ... by rl3 on Thursday February 20, 2020 @08:00AM (#22374746)
No, it's legitimate. Google is the last company that needs marketing gimmicks, let alone ones based on outrage.
Re:Cloud Vision API will not return gendered labels such as ... by codesternews on Thursday February 20, 2020 @04:21AM (#22373804)
Are we becoming more stupid?
[dead]
Re:Cloud Vision API will not return gendered labels such as ... by s3krit on Thursday February 20, 2020 @11:39AM (#22376292)
>effectively denies my biological masculinity

How does it do that? All they are doing is no longer using perceived masculinity as a metric for determining whether you might be male or female.

>I'll not bother making a fuss

And yet here we are

Re:Cloud Vision API will not return gendered labels such as ... by growlist on Friday February 21, 2020 @05:34AM (#22382197)
A post on HN is a bit different to the mind-melting full-spectrum 'there's no such thing as men and women guyz!' etc. subversion efforts afoot.
Re:Cloud Vision API will not return gendered labels such as ... by GreenJelloShot on Thursday February 20, 2020 @01:47PM (#22377495)
> And yet here we are

A two sentence post on HN is "making a fuss"???

Re:Cloud Vision API will not return gendered labels such as ... by foldr on Thursday February 20, 2020 @09:24AM (#22375247)
>In a way I find it offensive that this service effectively denies my biological masculinity

All it's saying is that it won't make gender classifications based on perceived masculinity/femininity.

Just because you can automatically classify certain features to some degree of accuracy doesn't mean you should. A company like Google also has an ethical obligation to think about how the technology will be used.