What if it was an 8x8 grayscale photo of their face? We wouldn't be particularly surprised that it can guess race. The fact that we struggle to detect patterns in the data doesn't mean they don't exist.

> We wouldn't be particularly surprised that it can guess race.

That's actually a great example of this problem, though.

https://www.theverge.com/21298762/face-depixelizer-ai-machin...

> It’s a startling image that illustrates the deep-rooted biases of AI research. Input a low-resolution picture of Barack Obama, the first black president of the United States, into an algorithm designed to generate depixelated faces, and the output is a white man.

> It’s not just Obama, either. Get the same algorithm to generate high-resolution images of actress Lucy Liu or congresswoman Alexandria Ocasio-Cortez from low-resolution inputs, and the resulting faces look distinctly white. As one popular tweet quoting the Obama example put it: “This image speaks volumes about the dangers of bias in AI.”

Wanting the AI to have the same racial bias as Americans, that is that Barrack Obama who is half white half east African should be categorised the same as someone with west African heritage is just dumb. Barrack Obama has a white mother and a father from Kenya, so has little resemblance to African Americans who have mostly west African heritage, only reason people don't see a difference is because they are so used to categorise people by skin color. Of course getting data from all areas of Africa and all mixes of people would be great, but there are limits and adding in more west Africans wouldn't have helped accurately depixel Obamas face.

This is actually pretty funny. In much of the American Deep South, even Americans from other parts of the country can't identify race reliably when sitting next to a person.

The notion that a classifier can reliably identify race based on an 8x8 grayscale is risible.

Your comment is the only one in this entire thread that mentions reliability. Even if it were a meaningful metric in the context of race, an AUC of .68-.72 is a puncher's chance at best.

That's 1/4 the size of the texture of a Minecraft block. An AI could easily be trained to identify the block type of a Minecraft block based on an 8x8 subsample of one of its textures, and there's no reason something similar couldn't happen with biological race. Unless you a priori assume all racial differences are socially constructed and/or only skin deep.

Every Minecraft block of a particular type is identical.

The same is... not remotely true for humans, or even two chest x-rays of the same human.