News for hackers. Stuff that matters.
Re:ProctorU is dystopian spyware by pdpi on Tuesday November 09, 2021 @11:49AM (#29164057)
It's important to note that none of these things are necessarily done deliberately (though "white supremacy" is perhaps a bad way to express "racism that benefits white people specifically"). Other than transphobia, either the linked article, or the letter linked in that article provide evidence for all of the accusations. Facial recognition software that doesn't handle dark skin well intrinsically treats different ethnicities differently, in this case disadvantaging non-white people. Many of the markers for "suspicious behaviour" that are used to detect cheating are also present in people with both mental and physical health conditions. Dealing poorly with headware has an outsized influence for non-white women (who are likeliest to wear headwear that obstructs the face).

Again, I wouldn't chalk any of this up to deliberate bias against any of these groups, but it's all bias anonetheless.

Re:ProctorU is dystopian spyware by jedimastert on Tuesday November 09, 2021 @01:02PM (#29165031)
This is speculation, but many transgendered folks tend to present gender in an non-traditional way, as well as people in the midst of transitioning that may be "in between" presentations in a traditional sense. If you only train on cisgendered faces, you may only be training on sort of "default" gender presentations, facial structure, and the like, which may give a similar disadvantage.
Re:ProctorU is dystopian spyware by pdpi on Tuesday November 09, 2021 @02:01PM (#29165905)
I can totally 100% see how transgender people might trip up some of these things. For the point of this discussion, though, all the other biases had a specific example in the linked article, whereas that one didn't.
[dead]
Re:ProctorU is dystopian spyware by rocqua on Tuesday November 09, 2021 @02:21PM (#29166171)
White supremacy is the idea that white people are inherently better and more capable and therefore more deserving. Hence, for something to be white supremacy, it doesn't just need to have a bias favoring white people. It should also have a justification of this favoring based on white people deserving better or being better.

Racist systems that have encoded societies biases are generally not white supremacist. The 'justification' for those systems is often things like "this is just the way it is" or "this was easier to do like this" or "I went with my own experience".

These days a lot of racism does not come from white supremacy. It either comes from something like familiarity bias of people in power, or from following the status quo mindlessly. Calling those acts white supremacist can be dangerous. It allows the real white supremacists to hide among the unknowing. It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.

Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @03:03PM (#29166732)
> White supremacy is the idea that white people are inherently better and more capable and therefore more deserving.

That's a common dictionary or encyclopedia definition of the white supremacy. More broadly, white supremacy also refers to the systems and structures of power that are built into most of "western" (a better term might be post-colonial) societies that favor both white people, and people who support or uphold the balance of power in those post-colonial societies.

> Racist systems that have encoded societies biases are generally not white supremacist

I agree, however I think that those racist systems that are not inherently white supremacist in nature are largely rooted in non-colonial countries (basically countries other than the European colonial powers, and the countries that grew out of those colonies).

> It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.

That is just not true. If someone does or says something racist, they can and should be challenged on it. If they become defensive, there are multiple reasons that could happen, but if the reason is that they simply didn't know better, it's just the way their society is, or if the reason is that they are opposed to "wokeness" (which is a catch-all for intersectionality, critical race theory, and many other modern perspectives and ideologies that are largely centered on dismantling power structures and reducing bias and discrimination), then it's likely that they are supporting white supremacy out of ignorance (whether that ignorance is from being uninformed or uneducated, or the more malicious willful ignorance of people who choose to use or engage in racist norms because they are opposed to "wokeness" from an ideological or other perspective).

Pushing people who do things that could be cast as unintentionally racist is the only way to a) educate them, so they can do better, or b) determine if it was an intentional act. I know this from practical experience, and it was only from going through the hard and painful experience of being called out on harmful "unintentionally" racist jokes and behaviour that I learned to do better after being raised by a family that had (and for the most part, still has) some pretty racist and discriminatory views.

Re:ProctorU is dystopian spyware by rocqua on Wednesday November 10, 2021 @04:21AM (#29172689)
Lets not devolve into a semantic discussion about white supremacy. Under your definition I agree with a lot of what you are saying.

I still think we should not tell people who do racist things "you are being a white supremacist". Instead we should say "that thing you did was a bit racist" or even "that thing you did could be seen as racist". We should _definitely_ challenge those people. But if you are trying to get someone to change their mind or behavior, you gotta be real careful about their feelings.

Calling someone a white supremacist is going to hurt their feelings quite a bit. And that isn't going to make them more likely to consider if they should change or did something wrong. That doesn't mean don't challenge people on their actions. That means challenge people on their actions very gently.

If we don't treat these people gently, then we lose the less introspective part of the population. They start getting reactionary, start 'banding together' against these annoying people. They start 'fighting back in the culture war'. Basically, getting this wrong is how you create republicans.

All of the above means I think we should be considerate in how we challenge racists. And I think part of that involves being really careful about the term white-supremacy.

Re:ProctorU is dystopian spyware by visarga on Wednesday November 10, 2021 @03:01AM (#29172228)
> the systems and structures of power

what a vague concept lumping together unrelated things

the systems of power in one place are different from another place, they are unrelated, can't be reified like this

Re:ProctorU is dystopian spyware by sophacles on Tuesday November 09, 2021 @05:14PM (#29168231)
There's an old saying: "It is difficult to get a man to understand something when his salary depends upon his not understanding it" (often attributed to Upton Sinclair, not sure if that's true or not tho).

Id like to suggest that what you are calling "familiarity bias" might have a component of the quote in it too. Not salary in this case, but social position. That is in the racist system, one race of folks get better treatment, and if they want to maintain better treatment, the status quo must be maintained. The group of people at top of a racial hierarchy (that is in the supreme position), are incentivized to keep the racist system. When race is considered a bad reason to judge a person, they still are incentivized to maintain the system, just find different words to justify the status quo.

I guess a different way of saying this is - white supremacy describes a race based social hierarchy where white people are at the highest level. It has also been used to describe the lowlife Nazi or KKK wannabes that advocate for it in the baldest terms, but they are bigots who advocate for white supremacy using racist terms like "inferior genetics" or worse.

Compare the term racist itself - there are folks who would have you believe that the term is limited to personal bigotry against people of a different race, and has nothing to do with the rules and actions of systems (a position I think you don't hold due to your description of racist systems).

Re:ProctorU is dystopian spyware by pdpi on Tuesday November 09, 2021 @02:36PM (#29166357)
I'm not handwaving anything away. It's just that I don't think all racism is the same.

White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.

This story is about a software system that (among many other issues) doesn't work well with darker skin tones in low light. Especially in light of all the other failure modes, I'd ascribe that to carelessness or indiference, mixed with pressure to reduce false negatives at the expense of more false positives. I wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.

Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @03:33PM (#29167076)
> White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.

That is only half of the story of white supremacy though. The other half is the entrenched systems and biases baked into those systems that largely benefit white people, and that train people, through experience, to prioritize preserving the existing systems and status quo.

Not considering the fact that there is a well documented history over the last 20 years of tech companies and business in general prioritizing the experiences of the white majority, at the expense of people of colour, is largely the reason why you can "wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.", and not consider that being the norm, or even acceptable as being indicative of white supremacy.

Those biases may not always, and only impact people of colour, but they do overwhelmingly benefit white people. That's the entire point of the article that OP shared, and the references the author of that post uses to back their claims.

Re:ProctorU is dystopian spyware by Prophasi on Tuesday November 09, 2021 @06:48PM (#29169147)
It seems clear that a dev team could whip up a product that tested well (using folks in office, family members, friends, etc.); was trained with datasets that - for whatever reasons - weren't sufficiently varied; and hit some mark of success and pushed it out the door to refine the rest later. It also seems clear that the resulting product could do poorly when recognizing black skin, due not to ill intent but lack of polish with the resources on hand.

But something I always wonder when accusations like "white supremacy" are thrown around: is it falsifiable? What evidence would dissuade you from that?

- What if both ends of the spectrum do poorly and extremely pale people have problems, too?

- What if the threshold is dark black and lighter-skinned black people, Asians, Middle Easterners and other non-white people are able to use it successfully?

- What if only a narrow band of light levels work, making it clear their testing range was generally too narrow, not just in skin color?

- What if they took care to incorporate black models in testing, but the photo quality (and their own in-house cameras and lighting) overestimated the quality of most home users'?

And what of the myriad other things that were done poorly in the software: limited OS support, bugs, excessive memory usage, overall intrusiveness, browser limitations, disallowed mobile devices, lack of multi-monitor support? Do they likewise arise from systematic oppression of some group? What if we dig in and find that white people are more likely to use iPads, Linux, and multiple displays?

Most often these accusations flow in only one direction, and that all other flaws or problems are taken to be simply happenstance and noise. Certainly anything that impacts white people negatively will not be automatically seen as anti-white, although in a world with activist devs, such a result isn't incomprehensible.

Claims of white supremacy (among other accusations of character) are thus, to my mind, wildly speculative and carry a very heavy burden of proof.

Re:ProctorU is dystopian spyware by jedimastert on Tuesday November 09, 2021 @01:04PM (#29165064)
I think "white supremacy" tends to imply direct, explicit bias, and may sort of exclude the built-in "unrealized" biases that exist in the current culture, where white supremacy was the foundation but not necessarily explicitly imbued.
Re:ProctorU is dystopian spyware by visarga on Wednesday November 10, 2021 @01:46AM (#29171832)
> how is it even possible to handwave away systemic racism and bias "that benefits white people specifically" as anything other that white supremacy?

Your way of grouping people by race is kind of arbitrary. It puts together rich and poor when they only share a skin color. How is white supremacy working in Bulgaria, for example?

Re:ProctorU is dystopian spyware by fabianhjr on Tuesday November 09, 2021 @12:59PM (#29164989)
[deleted]
Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @01:24PM (#29165322)
Yes, I totally get that, unfortunately I accidentally left out part of the first question in an edit :(
Re:ProctorU is dystopian spyware by fabianhjr on Tuesday November 09, 2021 @02:14PM (#29166073)
Ah gotcha, deleted the comment.