The episode about the black engineer who isn't detected by the motion sensors is basically straight out of HP's webcam fiasco[1](although the show takes it to the logical and hilarious extreme).
[1] http://newsblogs.chicagotribune.com/race/2009/12/hp-webcam-c...
https://www.hulu.com/watch/81765
In my opinion, BOT is one of the best comedy shows of the past decade and that episode is widely considered to be the best one.
https://www.theatlantic.com/technology/archive/2016/04/the-u...
Can be solved by illuminating the face with IR though, which should work across all ethnicities.
It's not that hard to find both conscious and subconscious examples of systemic or individual racism. Engineers taking the easy path with webcam facial recognition is probably not a good one and serves only to give more fuel to those who claim people jump to the racism cry too quickly.
Laziness and racism aren't mutually exclusive. Historically in America, white people haven't considered black people as fully human. You can read the various declarations of secession or the 3/5ths compromise for that. Or the long post-Reconstruction history of slightly more subtle ways.
Sure, laziness was involved here. But deciding a product was good enough to ship without caring that it worked for black people requires the effective belief that black people didn't really count as people. At least, not people that mattered. Imagine the reverse: if the product didn't work on white men, would it have been shipped? Of course not.
When laziness just happens to have a blatantly racist outcome in a place where there is a centuries-long history of racism, Occam's Razor suggests the explanation is racism. If laziness could not have caused the bad outcome to happen for white people, then it's pretty clear that pure laziness is not the real cause. It's instead white people being differentially lazy when it comes to black people. That's clearly racism.
As a confirmatory example, look at the American justice system. In a lot of places and times, the same nominal laws applied to white and black people. But they were enforced very differently. Serious crimes against black people were ignored. Minor infractions by black people were enforced vigorously. [1] Were the cops lazy? Sure, everybody's lazy sometimes. Was that why there was a racially different outcome? Definitely not.
[1] Examples of this are all over Loewen's "Sundown Towns" for example.
So plausibly, this really was an innocent and understandable mistake. Maybe they grew up in a town full of white people, and maybe all of their friends and coworkers are white. I understand what they did is technically racist, but that's because of that particular combination of people. Let's use _this_ 'experience' of "racist", but not it's exact definition. For example:
- It was racist you forgot your mother in law's birthday (who you see once a year.)
- It was racists you didn't lift the toilet seat. (Edit: I grew up in a culture where it was offensive to not raise the seat)
Definition: Any unintentional side effect from not thinking about someone* outside of everything you've experienced, and presumably causing harm to that outsider.
* Usually this person is disadvantaged in some way. But almost everyone is disadvantaged in some way, thus neutralizing this particular point IMO.
Does that sound rational? I don't think it does.
The world is full of suffering, and I'm advocating people help and love the people they are surrounded by. If many people did that, it would be easier to recognize each other, where ever you come from. Why? Because hate will push away everything it is unfamiliar with, but love will accept everything it is unfamiliar with.
(Now meta argument, do I expect people to change? No. So I try accepting them instead.)
> Maybe they grew up in a town full of white people, and maybe all of their friends and coworkers are white.
This is not an accident. All-white contexts in America are the result of personal and systemic racism. People who grow up in those contexts are, unsurprisingly, more likely to be personally biased and will regardless do things that further systemic racism.
People in that condition have a choice: they can either overcome their upbringing or they can continue to support white supremacy. If they do the latter, well, then they've made a choice. You can accept that if you want. I don't.
Less than 15% of the U.S. is African American so it's not at all surprising that there are areas where there are very few black folks as a result. That's nobody's fault and shouldn't be described as racist because, well, it isn't.
If you'd like to learn more, start with Loewen's "Sundown Towns", which describes how hundreds and possibly thousands of towns across the US were turned and kept white:
https://www.amazon.com/Sundown-Towns-Hidden-Dimension-Americ...
These were known as "sundown towns" because non-white people had to leave by sundown or face violence: https://en.wikipedia.org/wiki/Sundown_town
These came about not because there were never any black people, but because white people indulged in violent ethnic cleansing. This happened most prominently during the Nadir: https://en.wikipedia.org/wiki/Nadir_of_American_race_relatio...
He proves this quite clearly through census data, where places that used to be racially mixed suddenly got white and then stayed that way for decades.
He doesn't give any specific Vermont examples, but Vermont had a prominent Klan presence in the 1920s: http://vermonthistory.org/research/research-resources-online...
Later, as suburbanization happened, white supremacy was maintained more subtly, through means such as racial covenants: https://www.sevendaysvt.com/vermont/discrimination-in-deed/C...
It's also not necessary for a lot of white people to be actively racist to preserve white supremacy. As is clear from Loewen's book, all it takes is a small number of people willing to be active (e.g., making threats, burning crosses) to drive off new black arrivals. The majority can be passively accepting of a racist system without lifting a finger.
But that doesn't mean they won't end up biased. White people who don't know many black people are more likely to be biased, and all-white towns produce people that are more likely to have racist views. (For more, see Loewen's Ch 11, "The effects of sundown towns on whites".)
> But deciding a product was good enough to ship without caring that it worked for black people requires the effective belief that black people didn't really count as people.
Nobody ever said "oh it doesn't work with black people? Who cares LOL." I was going to make a crack about mental gymnastics but this isn't even that. You're just making shit up at this point.
> When laziness just happens to have a blatantly racist outcome in a place where there is a centuries-long history of racism, Occam's Razor suggests the explanation is racism.
No, even using your sentence here you admit the cause is laziness but then try to reframe it to racism because that fits your world view. Just like the fact that we're talking about webcams and you're talking about the 3/5th compromise and the American justice system.
Is there institutional/structural racism in the US? Absolutely. Webcams aren't a good example of it, sorry.
We don't know whether or not they knew it didn't work with black people before they shipped it. Maybe they did and dismissed it with technical hogwash, as you are here. Maybe they did and decided it was an ok flaw. Maybe they didn't know because the company doesn't have any black employees and nobody though to test it on black people because the people in charge don't know any. Or maybe they do and just don't care. There are many possible paths to this decision, but any of them demonstrates systemic racism.
And this is on topic, because the broader topic is social implications of technology.
As an aside, your attempt to narrowly define the topic so racism can't even be spoken of is a pretty typical move from people experiencing white fragility: http://libjournal.uncg.edu/ijcp/article/view/249
Speaking of work, have you tried testing your definition of racism on non-white subjects and products, such as Black Entertainment Television? It doesn't hold up very well.
You could mean two things by "testing your definition of racism on non-white subjects": in the US or elsewhere. Either way, yes, of course I have. When going elsewhere it's necessary to speak of "the dominant racial group", and that can be helpful here too. So taking BET, we can ask questions like, "What effect are they having on non-dominant racial groups?" And, "Do their actions tend to increase or decrease the racial power imbalance?"
I'm not much of a TV person, so I'm not familiar with their shows. But I took a look: https://www.bet.com/shows.html
My first impression is that they are having positive effects on a non-dominant racial group, and are acting to decrease the racial power imbalance. Therefore, no, they are not racist.
Since you are anonymous person with a new account turning up to be a dick about the topic of racism, I presume your question was entirely insincere. But just in case, I hope you find the answer useful.
It can be both. An attitude that considers facial recognition software acceptable if it only recognizes light-skinned people is both lazy and racist. Only choosing to test on light-skinned people is racist because doing so assumes dark skinned people are an exception or an outlier, rather than an equally valid part of the set of "human faces."
The assumption of ones own race being the default color for humans (rather than an arbitrary value) that leads to facial recognition software being trained on a data set so narrow that the resulting software can't even recognize skin colors that diverge from that norm is as much about implicit, albeit unintentional, racial bias as it is the particular technical problems involved and the need for engineers to cut costs and meet deadlines.
It's not entirely racist, but racism is a component.
I wasn't ready to admit the genius of the show before, but I am now: what you said is quite close to a line from the same episode (said by the sociopathic boss, Veronica) when told by the protagonist that the sensors were effectively racist "[...] it's actually the opposite of racist, because it's not targeting black people. It's just ignoring them. [company] insist the worst people can call it is "indifferent.""
There could be a hidden assumption on the part of the engineering team that light-skin is "normal" and anything else is a special case. Nobody is saying "I hate black people" or anything of the kind.
Or, as happened with a voice recognition system a former employer used, it was tested on the engineering staff, who happened to be all male. As a result it didn't work well for most women who tried to use it. There was no intentional exclusion of women from the test data, and I'd argue, no intentional exclusion of women from the engineering teams. But it is reasonable to say that systemic sexism that excludes women from engineering careers helped this system fail.
These kinds of problems are difficult to solve, because they aren't active decisions on anyone's part. They evolve out of pervasive conditions, and unconscious biases. At root, the source is still racism, or sexism (or another form of discrimination).
No, it's an insufficiently sensitive contrast filter combine with too narrow a training set. Screaming racism at everything that is even marginally approaching the topic of race detracts from real racism and ignores the actual issue here - shitty software.
If the system had failed for with people with different hair color rather than skin color would they have been equally lazy?
Personally, I think it could be understandable people didn't consider race when developing facial recognition technology, especially when we've only had mainstream awareness of this for under a decade and many people live in racially homogeneous or dominated cultures. However, I don't think it's acceptable for organizations, and the time when you can safely say you didn't understand biased learning data will be over soon. There are considerations you need to make scaling your tech from personal project to something the public will consume.
Also, the day will come when computers can point out racist stuff better than the average human can now, albeit with a high false positive rate. I say this because it's relatively easy; even if you only count a subset of tweets talking about racism as not trolling, that's still a shockingly high number of meaningful things about the world many people aren't seeing.
If it's not worth the trouble to see if your system works for non-whites, that's beyond lazy.
Like the soap dispensers that don't work for dark skinned people.
Don't take it seriously.
Most sci-fi has the problem of trying to model a situation with limited social scope. The fact that they never turn out true is because society has social safeguards to prevent it from happening that the writers don't model.
1984 is also a comedy.
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
She told us we needed to sign a corrected lease because there was no way to make the rent billing system charge us the lower amount.
We explained to her that the building’s legal obligations trumped what their computer would do. They figured out how to change the amount in their billing system.
"Make game of that which makes as much of thee." is a quote from a book thousands of years old (The Rubiyat by Omar Khayaam) but it is as applicable as ever.
My wife is a victim of the OPM breach and had someone apply for (and in some cases open) numerous accounts with her data. One of those accounts was with T-Mobile and after three months of phone calls and strongly worded bills we filed an FCC complaint. A T-Mobile rep called us two days later and the issue was resolved (or so it seems anyway, haven’t heard anything in the past few weeks).
That’s what I always do with shitty utility providers - it’s funny how the “computer says no” excuse suddenly disappears when they start to be out of pocket.
I recommend complying with the process, mentally account for the extra costs when you sign up, stick to defaults and avoid complexity whenever possible.
On the other hand nowadays companies do respond to complaints much better than a few decades ago if you file complaints through their websites. Taking it to social media may also result in quick and drastic measures.
By the time I arrive to this stage, I’ve already got a replacement lined up and all it takes is to flick a switch.
> hurt your credit rating
From experience, the problem eventually gets resolved and they cancel any black marks on your credit report. But personally I never cared about mine anyway.
Due process has favored debtors for some time now, but the information isn't published, leaving shady "credit cleanup" companies to charge hundreds to do what you can do in many cases simply by logging into a website.
If you think this is a bad thing, then it's an organizational problem and has nothing to do with computers.
For example:
- Evidence of abuse of a drug.
- A second prescription which interacts badly with the first.
- Changing health circumstances (pregnancy, some sort of deficiency, failing internal systems, etc).
The correct thing to do in a case like this is to stop and coordinate with the prescribing physician. Having a pharmacist look at some change compared to how things were done prior and have a patient tell them it's a mistake and they should ignore it without consulting the prescribing physician is almost never a good idea.
If the migraine was really bad (bad enough to cause tangible damage) maybe you should sue the doctor for damages, or if it wasn't that bad, report him/her and go somewhere else next time.
Clearly, some commenters are also trying hard to be robot-swappable. :)
If you can’t deal with corner cases, or have none than leveraging rules/software is the way to go.
I think of vending machines and parking meters as automation of simple tasks, but what to do when they break?
I think google tries this by trying to “automate all the things”. It works most times but when it goes wrong it makes it very frustrating to correct.
The two key points are that you are committing the crime yourself, not asking somebody else to, and that it is in fact only potentially a crime, and even then it's relatively minor, and could be argued strongly in court.
You are not asking somebody else to illegally sell you opioids (for example) with no prescription. From what it sounds like, nobody was in real danger, just pain, and you expect people to risk everything they've spent their entire life working for just to prevent an hour of pain for some stranger?
Not sure how suing is related to the comment you replied to. If a chemist hands out drugs to people without prescriptions and they get caught, they will lose their licence, be barred for life, lose their store, get a massive fine, and in some cases serve jail time. Nothing to do with suing.
[0] Relative to medical emergencies.
Depending on the jurisdiction, pharmacists aren't allowed to prescribe stuff or treat people on the spot. That's business of a (licensed) doctor.
Mistake? *chuckles* We don't make mistakes!
*crashing sound as an improvised plug for the new hole in the ground falls straight through*
Bloody typical! They've gone back to metric without telling us!When you design business processes like these, firing, hiring or managing people always keep in mind something can go wrong. Always include an abort button to either stop or roll back the process.
It's basically the issue we're trying to solve, where to draw that line?
It does not loose meaning because it can be aborted if stuff goes FUBAR.
To this observation I highly recommend E.M. Forster's The Machine Stops, first published in 1909(!).
[0] http://www.ele.uri.edu/faculty/vetter/Other-stuff/The-Machin...
That one has actually happened: http://bgr.com/2016/05/17/car-gps-mapping-directions-lake/
It's a little unfair, though. There was heavy rain, fog, and a second article [0] says it was also in the middle of the night. The photo there also does make it look easy to mistake as a road.
[0]: http://torontosun.com/2016/05/13/woman-follows-gps-ends-up-i...
I remember Dwight and Michael arguing in Michael's car over whether the GPS was right as it kept telling them to go right which was directly into a lake.
"Dunder Mifflin Infinity" - Season 4 Episode 3/4
I'm glad it wasn't me driving there at night and in the rain, I could see myself making the same very bad mistake.
When the orders come from a computer. Will any of that happen, will there be someone responsible?
Humans are very quick to abdicate their own moral authority, and it is the most repugnant human impulse that exists. If someone puts a gun to your head, and tells you to kill someone or die, you are still responsible for whoever you kill. Others might consider your actions understandable, even forgivable, but you're still a murderer and still one who valued their own life above anothers.
If you do something because you were "just doing your job", that's even worse. That's exactly identical to doing something for money. Which is usually looked very poorly on by society.
The danger I see is the stifling of potential human subjective decisions. That broad decisions will be made and carried out without a sanity/humanity check by humans.
I think measures should be put in place for a guarantee of human intervention in the case of situational anomaly or system error as well as an emphasis on redress.
Since automation scales and a human workforce doesn't, a guarantee of human intervention might not be practically possible.
Anyone have any ideas on how this kind of thing might be mitigated?
Reminds me of the Google Ultron IT guy story, from 2014 (I think? How do 4Chan/American dates work?)
I wonder what other examples there are of people blindly following technology - people driving into lakes because their GPS told them to, etc. Plus, as our society gets more and more dependent on these systems, we may lose out on the flexibility that human mediators and problem solvers once gave us. The human tendency to defer to authority may never be as terrifying as when that authority is held by an uncaring machine with a couple bugs.
What was once satire has become too real.
[1]: http://betteroffted.wikia.com/wiki/Goodbye,_Mr._Chips