News for hackers. Stuff that matters.
Re:ProctorU is dystopian spyware by anon7725 on Tuesday November 09, 2021 @10:39AM (#29163132)
> “It’s become clear to me that algorithmic proctoring is a modern surveillance technology that reinforces white supremacy, sexism, ableism, and transphobia. The use of these tools is an invasion of students’ privacy and, often, a civil rights violation."

Must all of the cards in the deck be played at each turn? Things cannot simply be bad/user hostile/privacy invading, etc.

Re:ProctorU is dystopian spyware by pdpi on Tuesday November 09, 2021 @11:49AM (#29164057)
It's important to note that none of these things are necessarily done deliberately (though "white supremacy" is perhaps a bad way to express "racism that benefits white people specifically"). Other than transphobia, either the linked article, or the letter linked in that article provide evidence for all of the accusations. Facial recognition software that doesn't handle dark skin well intrinsically treats different ethnicities differently, in this case disadvantaging non-white people. Many of the markers for "suspicious behaviour" that are used to detect cheating are also present in people with both mental and physical health conditions. Dealing poorly with headware has an outsized influence for non-white women (who are likeliest to wear headwear that obstructs the face).

Again, I wouldn't chalk any of this up to deliberate bias against any of these groups, but it's all bias anonetheless.

Re:ProctorU is dystopian spyware by jedimastert on Tuesday November 09, 2021 @01:02PM (#29165031)
This is speculation, but many transgendered folks tend to present gender in an non-traditional way, as well as people in the midst of transitioning that may be "in between" presentations in a traditional sense. If you only train on cisgendered faces, you may only be training on sort of "default" gender presentations, facial structure, and the like, which may give a similar disadvantage.
Re:ProctorU is dystopian spyware by pdpi on Tuesday November 09, 2021 @02:01PM (#29165905)
I can totally 100% see how transgender people might trip up some of these things. For the point of this discussion, though, all the other biases had a specific example in the linked article, whereas that one didn't.
[dead]
Re:ProctorU is dystopian spyware by rocqua on Tuesday November 09, 2021 @02:21PM (#29166171)
White supremacy is the idea that white people are inherently better and more capable and therefore more deserving. Hence, for something to be white supremacy, it doesn't just need to have a bias favoring white people. It should also have a justification of this favoring based on white people deserving better or being better.

Racist systems that have encoded societies biases are generally not white supremacist. The 'justification' for those systems is often things like "this is just the way it is" or "this was easier to do like this" or "I went with my own experience".

These days a lot of racism does not come from white supremacy. It either comes from something like familiarity bias of people in power, or from following the status quo mindlessly. Calling those acts white supremacist can be dangerous. It allows the real white supremacists to hide among the unknowing. It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.

Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @03:03PM (#29166732)
> White supremacy is the idea that white people are inherently better and more capable and therefore more deserving.

That's a common dictionary or encyclopedia definition of the white supremacy. More broadly, white supremacy also refers to the systems and structures of power that are built into most of "western" (a better term might be post-colonial) societies that favor both white people, and people who support or uphold the balance of power in those post-colonial societies.

> Racist systems that have encoded societies biases are generally not white supremacist

I agree, however I think that those racist systems that are not inherently white supremacist in nature are largely rooted in non-colonial countries (basically countries other than the European colonial powers, and the countries that grew out of those colonies).

> It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.

That is just not true. If someone does or says something racist, they can and should be challenged on it. If they become defensive, there are multiple reasons that could happen, but if the reason is that they simply didn't know better, it's just the way their society is, or if the reason is that they are opposed to "wokeness" (which is a catch-all for intersectionality, critical race theory, and many other modern perspectives and ideologies that are largely centered on dismantling power structures and reducing bias and discrimination), then it's likely that they are supporting white supremacy out of ignorance (whether that ignorance is from being uninformed or uneducated, or the more malicious willful ignorance of people who choose to use or engage in racist norms because they are opposed to "wokeness" from an ideological or other perspective).

Pushing people who do things that could be cast as unintentionally racist is the only way to a) educate them, so they can do better, or b) determine if it was an intentional act. I know this from practical experience, and it was only from going through the hard and painful experience of being called out on harmful "unintentionally" racist jokes and behaviour that I learned to do better after being raised by a family that had (and for the most part, still has) some pretty racist and discriminatory views.

Re:ProctorU is dystopian spyware by rocqua on Wednesday November 10, 2021 @04:21AM (#29172689)
Lets not devolve into a semantic discussion about white supremacy. Under your definition I agree with a lot of what you are saying.

I still think we should not tell people who do racist things "you are being a white supremacist". Instead we should say "that thing you did was a bit racist" or even "that thing you did could be seen as racist". We should _definitely_ challenge those people. But if you are trying to get someone to change their mind or behavior, you gotta be real careful about their feelings.

Calling someone a white supremacist is going to hurt their feelings quite a bit. And that isn't going to make them more likely to consider if they should change or did something wrong. That doesn't mean don't challenge people on their actions. That means challenge people on their actions very gently.

If we don't treat these people gently, then we lose the less introspective part of the population. They start getting reactionary, start 'banding together' against these annoying people. They start 'fighting back in the culture war'. Basically, getting this wrong is how you create republicans.

All of the above means I think we should be considerate in how we challenge racists. And I think part of that involves being really careful about the term white-supremacy.

Re:ProctorU is dystopian spyware by visarga on Wednesday November 10, 2021 @03:01AM (#29172228)
> the systems and structures of power

what a vague concept lumping together unrelated things

the systems of power in one place are different from another place, they are unrelated, can't be reified like this

Re:ProctorU is dystopian spyware by sophacles on Tuesday November 09, 2021 @05:14PM (#29168231)
There's an old saying: "It is difficult to get a man to understand something when his salary depends upon his not understanding it" (often attributed to Upton Sinclair, not sure if that's true or not tho).

Id like to suggest that what you are calling "familiarity bias" might have a component of the quote in it too. Not salary in this case, but social position. That is in the racist system, one race of folks get better treatment, and if they want to maintain better treatment, the status quo must be maintained. The group of people at top of a racial hierarchy (that is in the supreme position), are incentivized to keep the racist system. When race is considered a bad reason to judge a person, they still are incentivized to maintain the system, just find different words to justify the status quo.

I guess a different way of saying this is - white supremacy describes a race based social hierarchy where white people are at the highest level. It has also been used to describe the lowlife Nazi or KKK wannabes that advocate for it in the baldest terms, but they are bigots who advocate for white supremacy using racist terms like "inferior genetics" or worse.

Compare the term racist itself - there are folks who would have you believe that the term is limited to personal bigotry against people of a different race, and has nothing to do with the rules and actions of systems (a position I think you don't hold due to your description of racist systems).

Re:ProctorU is dystopian spyware by pdpi on Tuesday November 09, 2021 @02:36PM (#29166357)
I'm not handwaving anything away. It's just that I don't think all racism is the same.

White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.

This story is about a software system that (among many other issues) doesn't work well with darker skin tones in low light. Especially in light of all the other failure modes, I'd ascribe that to carelessness or indiference, mixed with pressure to reduce false negatives at the expense of more false positives. I wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.

Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @03:33PM (#29167076)
> White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.

That is only half of the story of white supremacy though. The other half is the entrenched systems and biases baked into those systems that largely benefit white people, and that train people, through experience, to prioritize preserving the existing systems and status quo.

Not considering the fact that there is a well documented history over the last 20 years of tech companies and business in general prioritizing the experiences of the white majority, at the expense of people of colour, is largely the reason why you can "wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.", and not consider that being the norm, or even acceptable as being indicative of white supremacy.

Those biases may not always, and only impact people of colour, but they do overwhelmingly benefit white people. That's the entire point of the article that OP shared, and the references the author of that post uses to back their claims.

Re:ProctorU is dystopian spyware by Prophasi on Tuesday November 09, 2021 @06:48PM (#29169147)
It seems clear that a dev team could whip up a product that tested well (using folks in office, family members, friends, etc.); was trained with datasets that - for whatever reasons - weren't sufficiently varied; and hit some mark of success and pushed it out the door to refine the rest later. It also seems clear that the resulting product could do poorly when recognizing black skin, due not to ill intent but lack of polish with the resources on hand.

But something I always wonder when accusations like "white supremacy" are thrown around: is it falsifiable? What evidence would dissuade you from that?

- What if both ends of the spectrum do poorly and extremely pale people have problems, too?

- What if the threshold is dark black and lighter-skinned black people, Asians, Middle Easterners and other non-white people are able to use it successfully?

- What if only a narrow band of light levels work, making it clear their testing range was generally too narrow, not just in skin color?

- What if they took care to incorporate black models in testing, but the photo quality (and their own in-house cameras and lighting) overestimated the quality of most home users'?

And what of the myriad other things that were done poorly in the software: limited OS support, bugs, excessive memory usage, overall intrusiveness, browser limitations, disallowed mobile devices, lack of multi-monitor support? Do they likewise arise from systematic oppression of some group? What if we dig in and find that white people are more likely to use iPads, Linux, and multiple displays?

Most often these accusations flow in only one direction, and that all other flaws or problems are taken to be simply happenstance and noise. Certainly anything that impacts white people negatively will not be automatically seen as anti-white, although in a world with activist devs, such a result isn't incomprehensible.

Claims of white supremacy (among other accusations of character) are thus, to my mind, wildly speculative and carry a very heavy burden of proof.

Re:ProctorU is dystopian spyware by jedimastert on Tuesday November 09, 2021 @01:04PM (#29165064)
I think "white supremacy" tends to imply direct, explicit bias, and may sort of exclude the built-in "unrealized" biases that exist in the current culture, where white supremacy was the foundation but not necessarily explicitly imbued.
Re:ProctorU is dystopian spyware by visarga on Wednesday November 10, 2021 @01:46AM (#29171832)
> how is it even possible to handwave away systemic racism and bias "that benefits white people specifically" as anything other that white supremacy?

Your way of grouping people by race is kind of arbitrary. It puts together rich and poor when they only share a skin color. How is white supremacy working in Bulgaria, for example?

Re:ProctorU is dystopian spyware by fabianhjr on Tuesday November 09, 2021 @12:59PM (#29164989)
[deleted]
Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @01:24PM (#29165322)
Yes, I totally get that, unfortunately I accidentally left out part of the first question in an edit :(
Re:ProctorU is dystopian spyware by fabianhjr on Tuesday November 09, 2021 @02:14PM (#29166073)
Ah gotcha, deleted the comment.
Re:ProctorU is dystopian spyware by acdha on Tuesday November 09, 2021 @10:48AM (#29163282)
Don't you normally expect people to make the strongest case for preventing something? It's pretty common for a lawsuit to bring every claim a lawyer can come up with on the hopes that enough will stick to get the outcome they want.

I would especially consider that in the United States we do not have a broad legal right to privacy but there are potentially much stronger tools available if that software skews negative outcomes towards protected categories like sex, disability, or race. From the perspective of a student, job applicant, etc. being asked to use this, if the legal risks cause an organization to stop using it they'll enjoy that as a win even if the outcome isn't a blanket ban.

Re:ProctorU is dystopian spyware by Enginerrrd on Tuesday November 09, 2021 @11:02AM (#29163483)
>Don't you normally expect people to make the strongest case for preventing something?

Not at the expense of grounded reasoning. When I see poorly substantiated claims, it shouldn't, but it really drags the whole rest of the argument down. The argument presented about lawsuits is actually a great example of why I think that's a broken system. They resolve that issue of lost credibility by considering each issue with total, clear, and mandated separation. Outside of that legal world with very well-defined rules, using such tactics reduces credibility.

I should note, that in this particular case, the claim of racial biases is at least substantiated by a believable anecdote.

EDIT: To clarify why I think the legal methodology is broken, its only because the same principles apply to criminal trials. IMO, prosecutors should NOT be throwing poorly substantiated charges at a defendant just to increase their winning probability and make the required defense more expensive.

Re:ProctorU is dystopian spyware by BobbyJo on Tuesday November 09, 2021 @11:11AM (#29163622)
> When I see poorly substantiated claims, it shouldn't, but it really drags the whole rest of the argument down.

It should. If someone is willing to make wild, unsubstantiated, claims, it should detract from their credibility.

Re:ProctorU is dystopian spyware by acdha on Tuesday November 09, 2021 @12:01PM (#29164222)
You're talking about a letter from a U.S. Senator citing published reports in e.g. MIT Technology Review and concerns raised by professional organizations. I think dismissing that as “wild, unsubstantiated” would require at least some discussion of the linked claims.
Re:ProctorU is dystopian spyware by BobbyJo on Tuesday November 09, 2021 @03:58PM (#29167383)
Why should I reference the particulars of the link when refuting a generalization?

If the parent comment had said something to the effect of "I should give the benefit of the doubt to a sitting US senator" then you'd have a point, but that context wasn't part of their statement.

Edit: Also, frankly, I wouldn't give the benefit of the doubt to a US senator. If anything, it makes the identity politics feel even more irrelevant.

Re:ProctorU is dystopian spyware by acdha on Tuesday November 09, 2021 @04:09PM (#29167501)
Okay, trying engaging intellectually with the reports rather than just reacting to someone’s one sentence summary of thousands of words. It’s kind of hard to see any definition of “identity politics” which includes the reports but not your emotional reaction to accurate words.
Re:ProctorU is dystopian spyware by BobbyJo on Tuesday November 09, 2021 @04:21PM (#29167638)
> Okay, trying engaging intellectually with the reports rather than just reacting to someone’s one sentence summary of thousands of words

I wasn't reacting to a summary, I was reacting to an independent premise. The sentence I reacted to, at least the way I read it, was broad to the point of being more of an axiom with which the commenter interpreted the post. As I see it, disagreeing with an axiom is an intellectual engagement. Feel free to counter my disagreement.

> It’s kind of hard to see any definition of “identity politics” which includes the reports but not your emotional reaction to accurate words.

Not entirely sure what you're saying here, but none of what I said was emotional. Simply pointing out that pattern matching is a viable way of filtering other people's thoughts and ideas. If someone makes wild claim, it should change the way you view other claims which may have seemed more rational in their absence. Not really an emotional statement in my mind, but again, feel free to point out which part of this you disagree with and I'd be happy to engage.

As it stands now you aren't really responding to anything I've said, but rather disagreeing that what I'm responding to warrants responding, which is rather tangential.

Re:ProctorU is dystopian spyware by tshaddox on Tuesday November 09, 2021 @11:27AM (#29163802)
If you think the accusations are poorly substantiated, then make that case, rather than just complaining that too many accusations were included in the same paragraph.
Re:ProctorU is dystopian spyware by isoskeles on Tuesday November 09, 2021 @11:41AM (#29163954)
I don’t have to. Using a Gish gallop of bad arguments doesn’t impose some moral imperative on me to prove every single one wrong.
Re:ProctorU is dystopian spyware by kelnos on Tuesday November 09, 2021 @11:47AM (#29164031)
No, but I think that raises the question of why you think the quoted bit above is a bad argument. It tracks with various reports I've seen linked in HN on this topic over the past couple years.
Re:ProctorU is dystopian spyware by isoskeles on Tuesday November 09, 2021 @12:52PM (#29164904)
I know that 95% of the time I see a Gish gallop of identity politics, it's not an argument to engage in at all because even when you do, you're called racist yourself unless you subserviently agree with every aspect of the argument. The identity politics argument is often a tempting one because it allows people to act righteously indignant and feel powerful.

Case in point, here's someone saying, 'How can you even handwave away systemic racism?' in reply to a comment agreeing that bias exists but is not deliberate: https://news.ycombinator.com/item?id=29164295

Also, the identity politics argument seems to hinder a simple, moral argument against surveillance software as a violation of privacy. The logical implication of the IP argument is that this surveillance software would be okay to use if we manage to work out all the kinks.

Re:ProctorU is dystopian spyware by ewalk153 on Tuesday November 09, 2021 @11:49AM (#29164049)
I have found that two to three strong points far out weight a list of 5-10 weaker points. This extends to the case when the original two points are included in a longer list.
Re:ProctorU is dystopian spyware by awillen on Tuesday November 09, 2021 @11:18AM (#29163713)
The difference is that lawyers act in a structured environment with specific rules on how things should be considered - you bring up all the possible claims because if any of them get thrown out, it doesn't impact the others.

It's not the same with general discourse - when you raise a bunch of issues that aren't especially relevant and seem designed to be inflammatory, you damage the credibility of your other arguments. Arguing that test proctoring software is transphobic is such a stretch that it makes you question whether they author has such strong biases against the software that their evaluation of it is just generally too biased to be trustworthy.

Re:ProctorU is dystopian spyware by smelendez on Tuesday November 09, 2021 @11:43AM (#29163979)
> Arguing that test proctoring software is transphobic is such a stretch that it makes you question whether they author has such strong biases against the software that their evaluation of it is just generally too biased to be trustworthy.

It seems like the software is matching people against existing images, based on the issue with the black student, and trans people are I would assume more likely to change their appearance, including as a result of taking hormones and having facial feminization/masculinization surgeries.

Re:ProctorU is dystopian spyware by kazinator on Tuesday November 09, 2021 @11:20AM (#29163727)
The rationally strongest case would be the privacy thing, not the neoliberal-outrage-inducing factors.
Re:ProctorU is dystopian spyware by cscurmudgeon on Tuesday November 09, 2021 @11:43AM (#29163985)
> Don't you normally expect people to make the strongest case for preventing something?

Yes, but with some evidence. Otherwise, as a society, this is a bad direction to head in.

Re:ProctorU is dystopian spyware by ygjb on Tuesday November 09, 2021 @12:02PM (#29164232)
I mean, it's an article about how a technology is negatively impacting marginalized groups, and cites research that backs those claims. It makes sense to cite the groups and practices most impacted by it.

You could read the research and refute it, or you could just bluster about things. I know you tried to expand on it in your comment below, but minimizing the specific concerns raised to "isms" and ignoring that at least two of the references in the articles linked and their references for the actual research addressed at least the socioeconomic portion of it, illustrates that you only applied your surface level perspective and criticism.

Re:ProctorU is dystopian spyware by aeturnum on Tuesday November 09, 2021 @10:54AM (#29163379)
Things are bad in particular ways - software that is more likely to falsely punish people of color does support the continued dominance of white people in a real, material way. Noting how things are bad seems useful and important to me.

That being said, there's a rotating list of "badnesses" that are in the zeitgeist and I agree that it's annoying to see them flogged at every opportunity (often w/o much insight).

Re:ProctorU is dystopian spyware by mercurialmaven on Tuesday November 09, 2021 @11:37AM (#29163917)
"software that is more likely to falsely punish people of color does support the continued dominance of white people in a real, material way."

The statement is at best extremely misleading, and at worse, mostly false. It also represents a juvenile, immature, and myopic perspective on reality.

I would suggest reading "Wealth, Poverty, and Politics" or "Discrimination and Disparities" by Thomas Sowell. He has been debunking the "inequality of outcome therefore racism" logical fallacy for decades.

Re:ProctorU is dystopian spyware by aeturnum on Tuesday November 09, 2021 @01:32PM (#29165459)
Where did I say that unequal outcomes must, necessarily, be caused by racism? What is misleading about what I said?

It seems like you're reading a specific thing I said about a specific scenario and universalizing it in a way you imagine I might universalize it. I'd love to hear a critique of what I actually said, or we could talk about our views of society in a wider way, but I can't respond to this combination of generally dismissing what I said and attacking what you imagine I might think.

Re:ProctorU is dystopian spyware by mercurialmaven on Tuesday November 09, 2021 @01:42PM (#29165579)
I stand at least partially corrected. I am not familiar with biases in facial recognition software but it looks like a real thing in some cases, caused for instance by lack of diversity in training data sets.
Re:ProctorU is dystopian spyware by monocasa on Tuesday November 09, 2021 @01:31PM (#29165445)
Software that's way more likely to fail you literally because of darker skin sounds way more like "inequality of opportunity" to me.
Re:ProctorU is dystopian spyware by mechanical_bear on Tuesday November 09, 2021 @02:28PM (#29166265)
It’s more likely to fail because…physics. The sensor on your webcam is only so big, and can only capture so much light. Darker faces require more lighting to capture details. Photography isn’t racist, it’s physical limitations that come into play.
Re:ProctorU is dystopian spyware by aeturnum on Tuesday November 09, 2021 @02:54PM (#29166619)
> Photography isn’t racist, it’s physical limitations that come into play.

I wanna pull this apart a bit because I think it's a good opportunity to talk about how systemic bias gets started. Digital sensor evolution is path-dependent. Technologists developed photo-sites that have "enough" dynamic range for most uses before moving on to increase the resolution on a sensor. What exactly is "enough" depends on your test data.

The sensor on a webcam is only "so big" as you say - but how that sensor balances resolution and photo-site count depends on what conditions they consider acceptable. We could build web cams that would see more pigmented faces better - there is no fundamental limitation in the technology itself. It's that a series of decisions have been made over years of development, generally without people thinking specifically about race at all, and we've arrived at a status quo that has adverse outcomes for people with different skin tones.

There was a similar process that happened with film photography[1]. Not that film, as a technology, is unable to capture dark skin - but that the development standards that were tested and distributed were designed for lighter skin.

Like, I agree that the webcams we have aren't intentionally 'racist.' But I do think that the status quo that has led everyone to accept this balance of dynamic range and resolution is reflective of valuing people with lighter skin more.

[1] https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b...

Re:ProctorU is dystopian spyware by MengerSponge on Wednesday November 10, 2021 @10:50AM (#29176003)
A closely related story that will also resonate with you: early microphones were tuned for male voices, which led to truly a lot of harm to women. It's just a matter of physics, as people are fond of saying. Mediocre technologists make mediocre products that were only validated for people like them.

https://www.newyorker.com/culture/cultural-comment/a-century...

Re:ProctorU is dystopian spyware by monocasa on Tuesday November 09, 2021 @02:34PM (#29166331)
A) The major issue is not the sensors, it's the lack of emphasis of ML training data for darker skin.

B) Even if it were simply physics at play, requiring the use of a system known to have physical constraints against darker skin causing failing grades purely on that metric is still pretty racist.

Re:ProctorU is dystopian spyware by compscistd on Tuesday November 09, 2021 @10:59AM (#29163449)
It’s hard to make people care with “bad/user hostile/privacy invading” because those terms have saturated descriptions of behavior that users are okay with. Example: tons of articles mention FB as a privacy invading or user hostile service but it continues to be used by people who don’t really care. Using the same terminology for something that is arguably worse with much higher stakes (algorithmic proctoring that “reinforces white supremacy, sexism, ableism, and transphobia”) is appropriate because it gets the reader to care by illustrating exactly what is possible with algo proctoring.

I sense you’re tired of discussions that mention the “cards in the deck”, likely because you aren’t affected by them and therefore care little for them. That’s honestly fair, but there’s value in writing that way to channel outrage into action.

[dead]
Re:ProctorU is dystopian spyware by kilnr on Tuesday November 09, 2021 @11:37AM (#29163919)
But extrapolating from one HN commenter onto the entire zeitgeist is acceptable to you?
Re:ProctorU is dystopian spyware by NikolaeVarius on Tuesday November 09, 2021 @11:45AM (#29164005)
Direct question. Do you agree with the aforementioned commenter and why or why not.
Re:ProctorU is dystopian spyware by tshaddox on Tuesday November 09, 2021 @11:25AM (#29163789)
Is your criticism that these accusations are false, or just that they included too many true accusations in the same paragraph?
Re:ProctorU is dystopian spyware by anon7725 on Tuesday November 09, 2021 @11:38AM (#29163922)
My criticism is that including a laundry list of "isms" is a polarizing, low-effort rhetorical device that elides a lot of nuance.

White supremacy: darker skin tones tend to photograph worse than lighter skin tones. Laptop webcams are notoriously crappy and can make this effect worse. Is this white supremacy or physics? Can the test instructions be modified to ask all users to have an appropriate lighting setup (ie lit from above and from the side to ensure that your face is foregrounded properly)

Ableism: for users who require assistive technologies, is it better to take a test in their own space with their own equipment or to travel to a test center and use shared lab equipment? For users with mobility challenges, is it better to be in their own space or travel to a potentially non-accessible testing center?

Sexism: for working mothers, better to take a test in your own home or travel to a testing center and arrange for child care?

Two "isms" seem more relevant but weren’t mentioned: ageism - because fuck boomers, right? Socioeconomics - not everyone has access to a PC that meets the specifications or can acquire one on short notice.

Lots of context is discarded when one engages in polarizing categorical rhetoric. I’m not here shilling for proctoring software but rather for nuanced discourse.

Re:ProctorU is dystopian spyware by tshaddox on Tuesday November 09, 2021 @11:48AM (#29164035)
> White supremacy: darker skin tones tend to photograph worse than lighter skin tones. Laptop webcams are notoriously crappy and can make this effect worse. Is this white supremacy or physics?

I’m not a photography expert, but it seems to me that cameras are physically equally capable of overexposing an image and underexposing an image. If a particular camera which is used in a facial monitoring system tends to do one rather than the other, I would ask why that is the case.

Re:ProctorU is dystopian spyware by frostburg on Tuesday November 09, 2021 @11:57AM (#29164171)
Doesn't work like that. Overexposing (with the same light) means having worse SNR due to higher gain.

The choice to use typically terrible cameras in a proctoring system disregarding that it might work ever worse than normally for a subset of people is suspect, yes.

Re:ProctorU is dystopian spyware by tshaddox on Tuesday November 09, 2021 @01:21PM (#29165287)
But wouldn't it be just as easy to create a cheap camera that tends to correctly expose dark skin in normal lighting conditions, and cannot dial down exposure enough to prevent light skin from being blown out in normal lighting conditions? If that's possible but cheap cameras tend to not work this way, then why is that not the case?
Re:ProctorU is dystopian spyware by frostburg on Tuesday November 09, 2021 @04:47PM (#29167940)
Yes and no, but mostly no. Lighter skin reflects more light (which means more signal), so it's inherently easier to image (it would be harder in extremely intense light, but getting fast shutter speeds is a lot easier than dealing with not having enough photons). Auto-exposure algorithms tend to work more accurately on lighter skin, too, which could be improved but is generally not something implemented at the hardware level in a webcam as far as I know (software can ask for different iso sensibility and shutter speeds).
Re:ProctorU is dystopian spyware by jpatt on Tuesday November 09, 2021 @12:19PM (#29164432)
The "isms" referenced are less about the fact you can work around these complaints line by line and more about the fact no one bothered to check them before rolling out required surveillance technology for education.

White supremacy is not just bad faith things done by bad people, it is also the assumption that whiteness is the default experience and the failure to account for that not being the case. Similarly with ableism and sexism.

That being said, complaining about the accessibility & inclusiveness of our required surveillance technology does have a dystopian feel to it, lol.

Re:ProctorU is dystopian spyware by greenail on Tuesday November 09, 2021 @12:39PM (#29164703)
> White supremacy is not just bad faith things done by bad people, it is also the assumption that whiteness is the default experience and the failure to account for that not being the case.

This only works with "new" definitions of racism. It is in fact plainly racist on it's face to demonize a group based on immutable characteristics. It is even worse when actual diversity of though is ignored and people of color are demonized because they don't agree with a race-marxist ideology.

Re:ProctorU is dystopian spyware by jpatt on Tuesday November 09, 2021 @02:21PM (#29166162)
I'm not sure I 100% understand what you are reacting to. I want to understand these two points a bit better:

  It is in fact plainly racist on it's face to demonize a group based on immutable characteristics.
Yes. Where have I, or the study's author, done this?

  It is even worse when actual diversity of though is ignored and people of color are demonized because they don't agree with a race-marxist ideology.
Yes. Where have I, or the study's author, done this?
Re:ProctorU is dystopian spyware by 8note on Tuesday November 09, 2021 @03:38PM (#29167132)
Would these webcams have shipped if you couldn't see white faces well on them? I don't think they would, and the test proctors would not think of requiring one that didn't. Both the camera makers and the proctors are putting in an assumption that these products are only/primarily for white people, and thus discouraging non-white people is not an issue.
Re:ProctorU is dystopian spyware by camgunz on Tuesday November 09, 2021 @11:46AM (#29164022)
It's not news that facial recognition is best at white male-presenting faces and bad at all the others. ProctorU is also pretty hard to use for the differently abled.

I think you just reflexively dismissed this because you saw a basket of words that normally go along with things you disagree with, but the argument is pretty solid.

Re:ProctorU is dystopian spyware by handrous on Tuesday November 09, 2021 @12:13PM (#29164360)
> It's not news that facial recognition is best at white male-presenting faces

Really? On the male part? I'd have expected it to do best with women, because I'd have figured facial hair is more difficult to deal with than a wider varieties of hair styles.

Re:ProctorU is dystopian spyware by monocasa on Tuesday November 09, 2021 @11:14AM (#29163657)
When they break even worse when you have black skin, it certainly sounds like a civil rights violation.
Re:ProctorU is dystopian spyware by LudwigNagasena on Tuesday November 09, 2021 @11:38AM (#29163931)
It really doesn’t.
Re:ProctorU is dystopian spyware by 908B64B197 on Tuesday November 09, 2021 @03:09PM (#29166793)
> Things cannot simply be bad/user hostile/privacy invading, etc.

In this case, the algorithm is actually bad for all ethnicities [0]. It's just that it's extremely bad for black students (fails half the time) and just regular bad for everyone else (fails a quarter of the time).

[0] https://www.theverge.com/2021/4/8/22374386/proctorio-racial-...

Re:ProctorU is dystopian spyware by captn3m0 on Tuesday November 09, 2021 @11:34AM (#29163889)
I followed the references to find the actual argument behind calling surveillance proctoring all of the above, and these are the relevant bits[0]:

>At the beginning of a test, these products ask students to verify their identity by matching their appearance with a photo ID. As Os Keyes has demonstrated, facial recognition has a terrible history with gender[x]. This means that a software asking students to verify their identity is compromising for students who identify as trans, non-binary, or express their gender in ways counter to cis/heteronormativity. If a student’s gender expression or name on their ID are different from their current gender expression or name, the algorithm may flag them as suspicious. When this happens, they may have to undergo another level of scrutiny to authenticate their identity, an already common and traumatic experience for trans and gender non-conforming students. If these students are not alerted of this possibility before the test begins, it may force them to either discontinue the test and risk their grade, or out themselves to their course owner when they may not want to, risking more trauma and discrimination including being denied financial aid, being forced to leave their institution, or have their lives put in physical danger.

>The Eugenic Gaze is a combination of white supremacy, sexism, ableism, cis/heteronormativity, and xenophobia. When we apply the Eugenic Gaze using technology, the way we do with algorithmic test proctoring, we’re able to codify and reinforce all of those oppressive systems while avoiding equity-based critiques because of our belief in the neutrality of data and technology.

Their recommendations are quite reasonable:

>Don’t use algorithmic test proctoring. Instead, focus on pedagogical techniques that you can use to design assessments, online or in person, that draw from personal experience or require students to apply concepts in unique contexts. If you have to use algorithmic test proctoring, make sure students know about the test settings and ID requirement well before they take a test, and assure them that you will not take any behavior flagged as “suspicious” into consideration that isn’t described explicitly in the syllabus.

The GP link[1] instead calls out "Facial Recognition Tech", and "Algorithmic Proctoring" as being too biased and follows up with a petition[2] to ban these entirely.

[0]: https://hybridpedagogy.org/our-bodies-encoded-algorithmic-te...

[1]: https://library.auraria.edu/news/2021/why-online-test-procto...

[2]: https://www.sheaswauger.com/post/petition-to-ban-facial-reco...

[x]: https://ironholds.org/resources/papers/agr_paper.pdf "The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition"

Re:ProctorU is dystopian spyware by agentdrtran on Tuesday November 09, 2021 @11:33AM (#29163874)
Which one of these is inaccurate?
Re:ProctorU is dystopian spyware by throwaway135790 on Tuesday November 09, 2021 @11:48AM (#29164045)
The claim of "transphobia" is extremely weak, for one. Clicking through the maze of links, the argument appears to be that because algorithms that attempt to guess gender based on photos sometimes guess wrong when trans people are involved, all attempts by computers to look at faces must therefore be transphobic, even if those computers do not attempt to guess the gender of the person whose face they're looking at.
Re:ProctorU is dystopian spyware by agentdrtran on Wednesday November 10, 2021 @12:27PM (#29177381)
Misgendering people intentionally is transphobic, and systems of this nature behave this way. For many trans people getting an ID to correctly reflect their name and gender is extremely difficult and can go for months/years without it being corrected; companies like Proctor U don't care about the problems this causes: https://www.teenvogue.com/story/exam-surveillance-tools-remo...
Re:ProctorU is dystopian spyware by vadfa on Tuesday November 09, 2021 @10:55AM (#29163399)
Re:ProctorU is dystopian spyware by comrh on Tuesday November 09, 2021 @10:55AM (#29163386)
The quoted person expands on their arguments on those topics in the linked article. If you think they're needlessly "playing a card" why not engage with the actual argument?
[dead]
Re:ProctorU is dystopian spyware by toqy on Tuesday November 09, 2021 @11:09AM (#29163585)
You're anonymous online. You have the ability to create a throwaway and refute the actual arguments in the article all you want with literally 0 repercussion. Honestly seems more like you're chasing your own version of woke points.
Re:ProctorU is dystopian spyware by NikolaeVarius on Tuesday November 09, 2021 @11:11AM (#29163608)
Ok Fine.

> It’s not clear to me that algorithmic proctoring is a modern surveillance technology that reinforces white supremacy, sexism, ableism, and transphobia. The use of these tools is an invasion of students’ privacy and, often, a civil rights violation."

Re:ProctorU is dystopian spyware by r00fus on Tuesday November 09, 2021 @11:06AM (#29163554)
b) is untrue. It's pretty clear when fake accusations are BS and you don't have to be "labeled" to call them out.

It does require you to engage with people directly.

Re:ProctorU is dystopian spyware by NikolaeVarius on Tuesday November 09, 2021 @11:12AM (#29163633)
Im calling out this accusation as fake and unsubstantiated.
Re:ProctorU is dystopian spyware by r00fus on Tuesday November 09, 2021 @11:18AM (#29163715)
You're not being accused of anything, so whatever's fake is in your head.
Re:ProctorU is dystopian spyware by NikolaeVarius on Tuesday November 09, 2021 @11:21AM (#29163745)
Clearly you are attempting to deliberately misread my comment since your comment literally makes no sense in context.

"This" clearly means the original accusation in the OP.

Re:ProctorU is dystopian spyware by named-user on Tuesday November 09, 2021 @10:55AM (#29163394)
Yes because to do otherwise would be exclusionist.