Perspectives differ on which one is more important. Personally, I’m absolutely for the former, but it increasingly seems I’m a part of the minority (or the silent majority).
If, for example, you made admission to elite colleges(which is a whole other can of worms) ignore ethnic backgrounds, you'd get a result where far fewer applicants of certain ethnicities would be accepted. In an arbitrary sense, this could represent "equal opportunity" if you believed that these people materialized from the ether the day they submit their application. But if you take into account their lives leading up to submitting their application, "equal opportunity" requires counteracting powerful structural factors that make it harder for people of some backgrounds to get into college.
The reason Harvard admission of Asian students is cited to such a tiresome degree is that it's a rare case where there isn't a clear upstream justification. That's why there's a big lawsuit and Harvard is getting tons of awful press about it. But it also lines up with the fundamental problem we're trying to fix with affirmative action. Harvard and its admissions systems were invented by white people, and they have proven to relentlessly favor white people, who then have easier lives and more money they can use to pay SAT tutors to help the next generation of white kids get into Harvard. It's not surprising that Harvard would warp the affirmative action system away from its original purpose, to correct for historical oppression, in order to... help white people get into Harvard. The fact that Harvard is discriminating against Asians is not a reason to implement a fake-fair system that has the end result of discriminating against all other non-white people.
It's theoretically possible that there are some other cases somewhere that show large inequality of outcome but have no unfairness upstream. But I'd rather have the task of discovering and fixing those rare cases than the present system, in which being born with a certain background gives you radical advantages or disadvantages on the basis of, basically, ancestral violence.
The point is, racist discrimination, even if “positive”, produces unfair outcomes. Obama’s daughter don’t need even more advantages. If you want to help kids from disadvantaged backgrounds (and I think that’s a worthy cause, even if you ignore “fairness”, simply because it increases the odds of finding the next Ramanujan and improves the well-being of the society), you should help people from disadvantaged backgrounds. Admitting them to degrees they don’t deserve isn’t actually helping them - it’s mainly just fixing superficial statistics and perpetuating harmful stereotypes (e.g. people not wanting black doctors because they suspect they’re worse because they weren’t subject to as strict standards as white, let alone Asian, doctors). To ensure actual equality of opportunity, you need to help them when they’re young, improve their school system, help their families, etc. Everything else is just a bandaid, at best masking rhe underlying problem.
Yes, that's very easy! Let me count the ways. First of all, Obama's daughters are much more likely to be murdered, abused or falsely accused by police than they would be if they were white. Even iconically famous black people are sometimes mistreated by police officers who don't recognize them. I'd say being at a high risk of murder counts as a disadvantage! I don't even need to draw a line back to ancestral violence for that one—that's just regular old present day violence.
Second, although the Obama sisters enjoy substantial family wealth due to their parents' incredible achievements, the fact that they were born black means that, statistically speaking, they would have been far wealthier if they'd been born to a comparably wealthy white family. For example, if the Obama's are the X richest African American family, they'd be far richer if they were the X richest white American family. Bingo! Disadvantage! Why does that wealth gap exist? There are many recent crimes, many tied to violence, that exacerbated the situation. But we can start by looking at the fact that Black people, including Obama ancestors, were violently forced to build wealth for white people without any compensation.
There are many more ways in which contemporary American society puts extra pressure and psychological burden on African Americans, yes even rich African Americans, all of which can be traced back to historical violence in our history, and all of which clearly affect the outcomes we associate with "success."
Whether you admit it or not, you're arguing that the person who "deserves" to win in our economy is the person whose ancestors perpetrated horrific crimes against his competitor's ancestors and who doesn't have to face the constant daily effects of ongoing racism in our society.
I don't think this is true. If you compare Obama's daughters against Bush's sons, probably. But against a random white person with no media exposure, unlikely.
> they would have been far wealthier if they'd been born to a comparably wealthy white family.
Again, why are we comparing against comparably wealthy white families? The reason why Obama's daughters were brought up in the first place is because we recognize that they are incredibly privileged compared to the general population, and they don't need more advantages. We don't care whether they are privileged compared to Bill Gate's children or someone comparatively more privileged.
Why would it be useful to compare the most famous and rich African Americans against "a random white person?" Is that the way we prove that African Americans are not disadvantaged, by comparing an anomaly from one group to the average of another? We are comparing comparably wealthy people in order to eliminate other forms of difference so we can see just the effects of ethnicity...
That said, usually the correct approach is not discriminating based on race, but discriminating on the actual "disadvantegedness" of the path taken by the applicant.
Accidentally that's how it's done around here for university (you get points for being disabled, poor, living far from the campus, and so on).
Changing the discussion to be structurally fair, not just transactionally, you still have the same impossibility issues arising if you don't have an oracle model with omniscient features. It's the same issues, but with a larger scope of features and outcomes.
Equality of opportunity seems much more organic/humanistic because it allows for the natural variation in people's values and personality
Even if we solve the funding problem it's still not actually equal opportunity because the poorer kids don't get the same support outside of school due to parents working longer hours at lower paying jobs because they got a bad education or living in a crummy house with lead paint (because that's all your parents could afford) or living in a town with contaminated water (Flint is just the one to grab headlines cities all over the country have abysmally high lead levels). Even if you want to blame community culture for some of these issues the question becomes where do you think that culture comes from, perhaps just maybe it comes from decades of lack of opportunity and systemic abuse?
How do you possibly hope to close the gap in our system where wealth almost invariably builds on wealth?
tl;dr: Equality of outcome doesn't mean everyone has to have the exact same exact life, it's just acknowledging our opportunities are directly tied to the outcomes of the generation before us.
Varies from state to state. In CA, only about 25% of K-12 funding comes from property taxes: https://ed100.org/lessons/whopays
I know of several school districts nearby that contain nothing but relatively new $1mm+ houses and abysmal school ratings. I would believe that there's some kind of complex nonlinear correlation between property values and quality of school, but it is very clearly not an easy "more expensive houses == better schools" calculation.
> perhaps just maybe it comes from decades of lack of opportunity and systemic abuse?
A fine point, and certainly not limited to any one race. I'm all in favor of helping those with fewer opportunities growing up, and I don't really care what race they are. If they happen to be mostly $RACE, it doesn't matter to me.
On your note though yes with unlimited money we could do that, given limited budget though targeting the assistance to those most affected makes sense. Also a program like that also does little to nothing for people who aren't very young when it's started. Also ultimately that's an equality of outcome style solution because it's giving additional resources to some.
(this is more about things like the Harvard affirmative action program) I think there's also a meaningful distinction to draw between inclusive discrimination like affirmative action and exclusive discrimination of groups. I know it's an extremely tough line to define at a government level but they feel like very distinct categories of activities.
The only distinction I see is in the rethorical spin that people make. A preferential treatment of a set of people is exactly equal to a discriminatory treatment of the complement of that set of people (e.g. positive discrimination of black people equals negative discrimination of non-black people).
Then that's where the energy ought to go. Adjusting the end result (e.g. by adjusting the college admission criteria based on race/ethnicity) doesn't really fix the root cause; it's a band-aid on America's broken leg.
If the starting conditions are unequal, then that is by definition not equal opportunity, so correcting those conditions should be Priority 1.
They are related, but "directly tied to" seems hyperbolic and deterministic.
Frederick Douglass is an obvious exceptional example of a human being transcending the outcomes of previous generations.
https://en.wikipedia.org/wiki/Socioeconomic_mobility_in_the_...
Is being raised a certain way tantamount to having 'less opportunity' to make better micro-decisions?
You're not solving the problem at all by attempting to sweep racism under the rug; quite the opposite in fact because it fails to address how race affects family background and discrimination in the first place.
You seem to have misunderstood my argument; maybe I wasn’t really clear in that part. I think a policy that helps lots of poor black people, some poor white people, and doesn’t help Obama’s daughters (because they don’t need help), is a good, fair, and most importantly effective policy. A policy that doesn’t help poor white people is racist. A policy that helps Obama’s daughters is ineffective (and I don’t want my tax money to go to that cause).
We have affirmative action policies and the like because we admit that some groups of people are disadvantaged due to a multitude of circumstances and we attempt to do things in such a way as to ensure that they can mitigate those circumstances.
Providing aid to low-income families is racist by the first definition, but not by the second. But (damn near) everything is racist by the first definition.
It would seem you are the racist here thinking a person needs support because he is black, not because he is poor.
In your example help isn't given to all black people, only to the poor ones.
If you consider help given to poor black people vs. help given to poor white people, it becomes proportional.
https://www.theatlantic.com/ideas/archive/2018/10/large-majo...
Edit: I will make it easier for you. If you do not understand most of this page[0] you are ignorant of one of the most fundamental debates in society. If you do you would understand my position. The wikipedia version might do as well [1]. It is ironic how people who don't "believe in outcome" are often ignorant.
[0] https://plato.stanford.edu/entries/equal-opportunity/ [1] https://en.wikipedia.org/wiki/Equal_opportunity#Theory
Arrow's theorem is a more domain specific and elegant proof regarding political properties that can be simultaneously held.
This sounds like a description of Harvard's admission policies with regards to Asians.
The answer is that asian immigrants are a self-selected subset of 1.5B asians in asia.
The social justice mental toolkit doesn't have an answer to this anomaly. American census numbers and the assumption of a normal distribution within them don't square with the highly-motivated subset of Asians who immigrated.
1) Cultural factors often raise people from poor to middle class
2) The above often happens with little monetary input
3) Culture is transmissible
My guess is a combination of social media obsession and limited life experience cause them to optimize for 'sounding woke' over 'things that help real people'.
I think this type of statement represents the main problem that folks who run down the “Asians at Harvard” rabbit hole.
People assume that Harvard admissions is all just based on who is the biggest brainiac based on grades and SATs, and that’s simply not the case at Harvard or any other elite school.
Decent article on the topic:
https://www.washingtonpost.com/education/2018/10/21/dockets-...
High SAT and high grades gets you merely a 2 (1 being best) in one of four categories (academic, athletic, extracurricular, and personal/leadership). If you get a 2 in three of four categories, it still only gets you a 40% chance of admission.
That said, getting a 1 in any category gives the applicant a bump to anywhere between 48% and 88% chance of admission. Note that only about 100 people per year get a 1 on academic, and that still only gives them a 68% chance of admission if that’s their only 1. Standards are sky high.
To anyone who has high scores and great grades in a rigorous curriculum and wants to apply to an elite school, I advise them to differentiate themselves in ways other than academics — it’s a much easier path to acceptance. In my opinion, these non-academic selection criteria is one of the things that makes top schools amazing places.
Specifically:
- Great grades and great SAT scores (750+ in Math and Verbal) are a baseline for being considered.
- The more of the following list below an applicant can check off, the more likely they are to get admitted. If they have several high quality checks, the requirements for grades and scores can actually decrease to a surprisingly low level:
(rough order of importance... non-exhaustive list)
1. Recruited athlete.
2. Diversity candidate (race).
3. Notable academic achievement (e.g., publish a paper).
4. Demonstrate leadership via some verifiable and substantial project.
5. Be a skilled-but-not-recruited athlete (esp. in non-varsity sports), entertainer, or person of interest (e.g., children of famous or powerful people). Accolades or championships help.
6. Create a substantial and successful business or non-profit.
7. Diversity candidate (geographical).
8. Have an incredible personal narrative (e.g., Malala) — this might be more important... it’s hard to tell.
9. Be the child of an alum.
10. Be the child of a substantial donor (also hard to rate in importance... it really only comes into play if they aren’t admitted normally).
11. School/department applied to (depends on university).
It boggles my mind when someone (Asian or otherwise) points to great grades and great SAT scores and gripes about not getting into an elite school. Elite schools explicitly state that they are looking for well-rounded applicants or “multidimensional excellence”. Why is that person/applicant surprised when they only focused on one dimension and did that in a non-exceptional way (compared to other applicants)?
Anyway, this is a topic I know quite a bit about. Please feel free to ask follow-up questions.
I'm not saying you're wrong, but when their internal paperwork seems to indicate an ASTOUNDING difference in "personality" for blacks vs Asians, at a large-numbers level.. I mean do we really buy that? Can we even just consider for a moment how racist that sentiment is? Against both races?
2. Affirmative action is a thing. If you stop the so-called “Asian quota” (e.g., by going race blind), then any slots gained will most likely result in losses for slots from other ethnic minorities. Some people believe that shaping the student body a certain way is good for the university. I personally agree with that general statement, but the specifics are not easy to hash out — there are many trade offs, and it’s not easy to determine where the magic cutoff is for too few or too many.
3. If Harvard goes race blind, don’t believe for a second that it will turn out like Stuyvesant or Berkeley. IIRC, when the change happened, Stuyvesant did not focus on the “multidimensional excellence” that most other elite schools do. Berkeley just lets in too many people to need to review to that level of detail. Harvard and other elite schools might see a very small Asian bump, but my guess is that it won’t be what people expect.
Note that I strongly advocate for fairness in the admissions process for all people, and I definitely think that some Asians get stereotyped poorly. That said, I also think that a lot of people, Asians as well as non-Asians, simply don’t make efforts to shape themselves in a way that optimizes their chance for elite school admission.
I can't tell if you're disputing my assertion that they first decided what their racial buckets "should" look like, then stack ranked within each bucket and fudged the numbers to make it look less discriminatory.
It's that last part that I have a real problem with. If Harvard wants to say, "we want x% black and y% Asian, etc".. ok, I guess, but if they're going to say "black people have lots of personality to make up for their lack of science skills", and vice versa to Asians, that's just racist as hell to everybody involved. Own your decisions.
That can be quite different for grad schools, though.
I actually had extracurricular activities in mind as one of the "seemingly-objective-and-reasonably-sensible metrics". (It can be harder to judge—e.g. "president of the choir club" could mean anything from "four people got together twice a month to sing" to "organized concerts and musicals in collaboration with the theatre department"—but maybe candidates describe more precisely what they did.) Legacy and athlete stuff make some sense in terms of the university's financial self-interest... most of the other things can be interpreted as demonstrating ability in some way (academic and leadership ability), or motivation/"Big 5 trait Conscientiousness". The "incredible personal narrative" seems like a wildcard that could point to lots of different things, but in Malala's case... in general, public relations could be a reason to take famous people. Department applied to, I assume that's because some departments have lots more applicants or don't need as many people than others. It's the racial and geographical diversity things that seem out of place.
Geographical diversity question: I've heard that, specifically, colleges like to have at least one person from all 50 states. Why is that? Just because it sounds cool? Is it really for some kind of viewpoint or background diversity? (A counterargument to that, at least for elite schools, is the same as for racial diversity: if you think the people from e.g. South Dakota who make it into an elite school are at all typical of South Dakotans, you've got another think coming.) Is it so they'll have alumni everywhere and therefore recruiting help and alumni interviewers everywhere, which wouldn't be efficient if all their alumni were in one place? (How many South Dakotans who go to an elite school actually stay in South Dakota after graduation? That doesn't seem to make sense.) A quick google merely turns up the quote "“Every school likes to be able to stick that little pin on the map and say they have kids from all 50 states,” said Andrea van Niekerk, a college admissions consultant and former associate director of admissions at Brown University", which is basically "because it sounds cool". (I also note that "a student from every state" could mean, at most, accepting 49 students you wouldn't have otherwise—probably less—whereas the extra 19% of Asians that Harvard didn't admit (I mention 42% vs 23% figures later), of a class of 1600, means accepting 300 students they otherwise wouldn't have—so the every-state policy has a smaller impact than the racial-diversity policy, and maybe the impact is small enough that indulging that whim is ok.) Do you know the rationale, for this and/or for geographical diversity in general?
Historical question: I've heard that the reason extracurriculars were originally introduced, as a criterion for candidates, was as an excuse to accept fewer Jewish students. Here's an article asserting this for Harvard specifically: http://web.archive.org/web/20190109180814/https://www.econom... Do you know if this is true?
SAT ceiling question: The maximum score you can get on the SAT is low enough that there are plenty of people that max it out, and plenty of others who would have maxed it out except they made one or two silly mistakes. I suspect that's why you say 750+, in that differences between anything above that are just noise; and that this still leaves a large pool to sort through. (As a data point, would the set of perfect-score SAT applicants still be too large to fit in a Harvard undergraduate class?) If the SAT had a higher ceiling, say going up to 900 or 1000 somehow, where the difference in actual ability between the average 850 and the average 750 was the same as between the average 750 and the average 650, then do you think they'd weigh SAT scores more heavily?
Olympiad question: are academic contests (math olympiad, physics olympiad, etc.) counted heavily in the "notable academic achievement" category? If so, would you say they substitute for a higher-ceilinged SAT—that if the latter existed, then they'd put less effort into researching and evaluating the olympiad contests?
Caltech question: I've heard that Caltech, at least as of some years ago, basically just selected for academic scores, and they had 0.6% blacks. Looking at this year, they have 40% Asians, compared to Harvard's 23%. Come to think of it, I've also heard that the University of California system is not allowed to implement affirmative action, and this year ... they break "Asian" down into subgroups, but if I add together "Chinese, Filipino, Japanese, Korean, Other Asian, South Asian, and Vietnamese", I get 42%. Anyway, any comments about Caltech or UC?
If you re-read my comment, it's clear that I'm not talking about rates of representation. You are trying to bring that in. We're talking about the process, be it a policy implementation or an algorithm.
It's the age-old "equality of outcome" vs. "equality of opportunity."
https://archive.org/stream/HarrisonBergeron/Harrison%20Berge...
EDIT: I didn't say this was just or right, just the current state.
In terms of regulations, I think by regulation they shouldn't be allowed to give the algorithms anything but loan amounts, payment schedules, and payment history. Don't give them locations. Don't give them age, name, race, or gender. Don't even give them the names of banks, since this too can be used in a discriminatory way that can be used to imply geographic location and thus race etc. Yes, people with no credit history will get terrible scores, but that's the point of a credit system. It should be based solely off of your merits and what you've actually done, not off of what people in your area/situation tend to do based on statistics. Then it's just statistics creating statistics, and socioeconomic mobility freezes
Getting loaned money is not a right, and if you are irresponsible, it's harmful.
So if lending more to a certain segment of the population hurts the bank's profit, I'd also expect it to cause more problems for that population segment too.
Just as different cultures value these differently, it makes sense that one culture that changes over time might as well.
That's really unfortunate.
I could go on quite a bit about how, as a man of color, it gets fatiguing having certain conversations or trying to express certain points of view in these types of socio-political debates lately (to my dismay sometimes right here on HN-a forum that holds itself to a certain standard of debate but seems quite eager to throw it out, convenience permitting), but looking around at a lot of other comments in this very thread that seem to run contrary to whatever personally defined "truths" are out there, it'd probably just get downvoted and flagged as well.
I'll just leave it like this, if it seems "shallow", to you-dang-then probably it's time for some deep personal introspection because like gdy, I think shadowbained's point isn't wrong and I don't fault him for the fatigue that's probably in his soul which results in a delivery style that doesn't fit a mold of pretend decorum. A mold manifested and curated by political-egotists, IMO. (This last point made as broadly as possible)
I'll try not to let it be too much of a burden on me as I sleep tonight..
I think the article encompasses your objections, to paraphrase : "algorithms can never be perfectly 'fair', because logic itself can never be perfectly fair"
Can we even define 'fair', perhaps we just know unfair when we see it. We seem to be seeing more of it - poverty, homelessness, racism, fact/science-denial, gerrymandering, police violence, gun violence, etc.
Yet, we should still strive for fairness.
There is a lot of ground between where we are now and 'perfect' - and this is AOCs point, essentially - our system of [self?]-rule is palpably unfair for most people, yet we are busily encoding more of that unfairness into algorithms with little oversight.
AOC [ and Bernie, and others ] want us to seek a better implementation of society, and encode that in our algorithms and laws - fairer voting laws, fairer district maps, fairer pay, fairer penalties. Or the converse - encode more fairness, as a means to realize it.
For example, it may be a hard math problem to design optimally fair district boundaries, but its noncontroversial to suggest we can reduce gerrymandering significantly.
One problem seems to be that most people equate socialism with communism [ 'evil' ], when in fact you are always picking from mix of Democracy, Capitalism, Socialism, Oligarchy etc. After Bernies campaign, and a couple years into the Trump experiment, people are now starting to listen to these old but useful ideas.
Ultimately if we don't share any wealth with the poor we will have a bloody revolution, and if we don't reduce Carbon emissions then no wall will keep out the displaced millions of climate refugees.
A bit more Socialism in our Capitalist/Democratic/Oligarchic/Socialist mix, is sorely needed right about now.
In my opinion, I think this idea is wrong, at least in the context of designing AI within contemporary American society. To frame "procedurally fair" and "representatively fair" as opposing value systems is a misunderstanding of the best arguments for a "representatively fair."
Let's use a common analogy: a race. This distinction imagines a situation where one race puts a bunch of runners at the starting line, tells them to run, and declares a winner based on who crosses the finish line first. That race is "procedurally fair" because all the participants raced under the same conditions. Then, there's a "representationally fair" race, where runners from "protected groups" are allowed to start the race from the middle of the track.
I think there are two ways that this isn't right that come from a common misperception, that the only relevant time to think about in the context of these decisions is the present. (I also want to mention that there's a lot of heavy-handed language use in this presentation that reveals the author's perspective. Algorithms tilted against black people "reveal" disparities,
First, it's important to have a more accurate perspective of the past. In most cases where "representationally fair"-type solutions are used, there is a historic reason why the "protected group" can't run as fast as the other group. If people in Hyderabad are more likely to defraud a micro-lender, that's not a spontaneous result of inherent differences between Hyderabad-type humans and other humans. A quick google search tells me that as of 2017 Hyderabad has the second largest poor populace in India. So, in the race to a successful economic outcome from birth, the so-called "procedurally fair" solution actually means taking the Hyderabad runners as infants very far back from the starting line. All else being procedurally equal, when the starting gun of "applying for a micro-loan" goes off, the Mumbaikars and Delhiites are already far ahead of the Hyderabadis. Once we have a more accurate perspective that includes the past, in order to make the race procedurally fair we have to move everyone into place at a common starting line, which will inevitably mean helping the Hyderabadis forward.
Second, it's important to acknowledge the effect of biased algorithms on the future. Not only will the unfair, pseudo-"procedurally fair" approach unjustly disadvantage some runners in this race, future races are calibrated according to achievement in past races. If you win one race, you get a head start in the next one. Micro-lenders who refuse to lend to Hyderabadis will exacerbate the relative poverty situation that the algorithm is picking up on. It's easier to pretend to yourself that you're making an algorithm that peeks into the world, makes an objective judgment, and then pops back out of the world. But in fact, people designing algorithms have a responsibility for the outcomes of their algorithms. If an unfair situation exists (for example, that just by being born in Hyderabad and not Delhi, any given person will start life with less economic power), your algorithm's consequences will either be helping to fix that unfair situation, or making it worse.
So, if your algorithm punishes Hyderabadi applicants for being poor, it is both unfair in the simplest sense once you account for where the applicants started and will increase the unfairness of any future round of applications. Put another way, what we're talking about at a high level is values. In one libertarian version of society, the purpose of the algorithm is to maximize profit for Simpl. That society values maximizing profit, and enshrines "shareholder value" as the centerpiece of its ethics. In AOC's socialist version of society, justice, equality, and eliminating poverty are valued highest. This is the broader point she's making about algorithms, that they reflect the values of the people who make them. Given the power that these algorithms have, in our example to lift people out of poverty or to deepen economic inequality, society and not just technologists (not the most diverse group in countless ways) should lead the decisions about what should be valued.
We're seeing more and more the consequences of a techno-libertarian approach, and I'm curious what we'll see as consequences if a more socialist approach wins for a while. I value fairness over shareholder value, so the prospect excites me. But I'm sure there will be lots of unintended consequences in such a system too (the over-cited "asians applying to elite colleges" example being a good case of an outcome that's not easily justified on its own. I think elite college admissions are more fundamentally broken, but that's another thread!)
Maybe???? But it seems very dishonest.
I think, if anything, the models are probably too true for our tastes as social creatures who evolved to avoid social conflict (being 100% honest with your unruly neighbors can lead to a loss of harmony that is much more damaging than just putting up with some disturbances). Our idea of "unbiasing" the models is more likely actually making them biased in a way that pleases our monkey brains.
Machine learning is going to be very good at identifying proxies for features we don't like, even if we don't directly provide those features. Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc.
If we want machine learning to ignore these kind of things it's probably going to have to be a separately weighted training goal that racial groups and so on need to have equal outcomes in the final results.
That's really what AOC is going for here. The black box will happily learn racist ways to weight things all on its own, we can't just throw up our hands and say oh well, if the algorithm is denying black people for home loans then we just have to accept it.
Because, in the short term, being racist is an effective greedy strategy. It's not your business's problem that, say, black people have higher rates of default (or whatever, example). But it also leads to bad social outcomes in the aggregate, and we've agreed that shouldn't be a factor in home loans. And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics.
Off-topic-ish, but only about 10% of Irish people have red hair.
This is not wrong, but it boils down to "don't make mistakes when optimizing functions". It's offensive because it's coming from someone who has nothing to contribute except "do your job better". It's like saying we have a problem with planes crashing, so engineers have a moral duty to build better planes. As though they weren't trying to build planes that stay in the air already. You can make serious sounding statements, or you can impose fines, or you can randomly pick people to publicly shame, but none of those things are positive contributions to hard problems.
Mechanical engineers are taught from the beginning of their careers to think carefully about the consequences of their decisions.
Computer engineers, not so much. We're taught how to optimize, and make machines that optimize, and make optimizers that optimize our optimizing machines, but it's pretty rare to encounter a data ethics course, and they're almost never program requirements.
Facebook got in trouble for algorithmic redlining in 2013, serving loan sharky ads to black people (which its algorithms learned via proxy features, like zip code). Does Facebook hire third-rate engineers who would point the engine on a plane the wrong way? No! Facebook has some of the best people in the industry! But even the best people in the industry were and in many cases still are not fully considering the consequences of our designs, because we weren't trained to.
Wrong. We had two courses at my school. From the ABET Criteria for Accrediting Engineering Technology Programs: "(e)Include topics related to professional and ethical responsibilities".
The issue here is proxy features, and that it isnt as obvious if your statistical model is doing something wrong. You identified zip code and race. What about number of cats? Would you have a particular gender and marital status in mind if a given data sample had a large value in the numberOfCats column?
So you ban making decisions based on number of cats. But amountOfKittyLitterPurchased, numberOfCatPicturesViewed, numberOfDislikesOnDogPictures, etc could be used as a proxy for numberOfCats. Try to come up with an exhaustive list of all features which could be used as a proxy for some protected class. Starting with politically incorrect jokes will get you part way there, assuming you possess an exhaustive list of all politically incorrect jokes. But you will need to be pretty creative to come up with more.
https://www.abet.org/accreditation/accreditation-criteria/cr...
As a software person, I have absolutely no issue with telling other software people to do their job better, and I see no shortage of people that aren't even trying on that front.
Its your comparison between a process that takes safety, reliability and a reduction of harm as a core, first value, and a process that cares first about money, and second about novelty and somewhere way down the line people, maybe, that is offensive frankly.
If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the first place.
Most people here are male, were once (or are) under 25 and likely drive. How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? Quadruply so if there are prior citations/accidents?
Shit, with a spotless driving record, my own insurance went up when someone t-boned me. Other driver was at-fault and I subrogated their insurance; didn't claim with mine. But I'm being lumped into a higher-risk pool with my own insurer for factors completely beyond my control-- because to some algorithm, being in an accident means I'm more likely to be in accidents, and all drivers who are involved in collisions are higher risks than drivers who aren't. Note how the fact that I wasn't at fault isn't factored in. This makes sense to you?
The social training data in much of Europe is skewed towards hating Gypsies and anyone perceived to be one. But it's the same as the ex-con problem-- deny them jobs, beat them, chase them away, and of course an ex-con or young Romani will commit future crimes. You're shaping behavior to validate your training data instead of making unbiased ("fair") predictions. It's the equivalent of betting on a horse, injuring its competition, then patting yourself on the back for being so goddamn good at this game.
You can't look at the world in terms of sheer numbers. The numbers themselves are dishonest.
isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.
Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident.
The latter is more likely to be wealthy enough to be able to pay cash for repairs and keep the entire incident off the radar. No insurance claim, no citations, no evidence. The 18-year old has no choice but to make an insurance claim, go to court and take points on their license.
Can we truly conclude that 18-year olds with red Mustangs are worse drivers than their elder counterparts?
(And yes, I realize that in the case of insurance, it's in their interest to minimize claims so in that context this is a valid conclusion. But this same skewed data gets shared outside of the industry as well.)
this is not a fair example; by assumption we have much more information about the 58yo than the 18yo. remove the bit about the midlife crisis and there's no reason to think the 58yo is anywhere near the same risk to insure. as a group, young people are less experienced drivers, have less facility for judgement, and the males probably drive more aggressively.
unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this.
and by the way, I'm a young male who drives a fast car.
I don't disagree. In this context, if older men make fewer insurance claims, they're better customers. On paper, he's less of a risk (to the insurer). In reality, they're both risks to other drivers. But only the 18-year old will be held accountable.
But assuming the under-25-and-male crowd are the worst drivers on the road based on number of insurance claims is not necessarily true, yet it's the basis for this oft-repeated assertion. That's my point. What isn't the insurance industry's data telling us when we use their data to make these conclusions? To get more accurate numbers, you need to be looking at the demographics of all body shop customers and the nature of their repairs. What demographics get the most front-end work done that isn't covered by an insurance check?
This new age of computing seems to be to make machines understand how the world works by putting blind faith in statistics, without ever stopping to question where the stats came from or how accurate they really are. The numbers aren't handed to us on golden tablets carved by God himself-- we get them from imperfect systems designed by imperfect humans. And then we teach dynamic algorithms how reality works based on this data and sell them as solutions, which we trust absolutely.
> and by the way, I'm a young male who drives a fast car.
No offense intended. I once was, too.
if you are just using it as an example of a misleading dataset, we are probably in agreement.
a better argument would be something like: suppose we detect a gene that indicates probability 1.0 of disease that causes the person's left arm to fall off.
it's then not a matter of "insurance" but a matter of "are we as the pool willing and able to pay for this?" recall company needs some profit as well as cost of coverage to maintain overhead so it's a different argument now.
And changing the algorithms isn't the right way to fix those external factors. If you try, their self-reinforcing nature is going to cause exactly the opposite of what you want.
To stay with the crime vs. race example, assume that the ratio of black to white criminals is greater than you'd expect from the ratio of blacks to whites in the overall population, and that is due to self-reinforcing external factors. Your algorithm for deciding whether to parole a prisoner picks up on that statistical regularity and rates blacks riskier than whites. Because you want to achieve representational fairness, you make the algorithm race-blind to equalize the rates.
What happens? The black prison population shrinks. But your algorithm wasn't wrong when it predicted higher recidivism risk, so the black crime rate rises. Your algorithm now requires an even stronger correction to remain race-blind. Black criminals realize that they have an easier time getting paroled, so the black crime rate rises some more. Debiasing the algorithm has made the situation worse.
In general, when you find out that your machine-learning model has some undesirable bias, the correct response is not to blind the algorithm. Instead, you should fix the data-generation process (the real world) until your model no longer picks up on the bias.
Edit: For a mathematically precise description of how attempts to make an algorithm more fair cause worse results when you consider the feedback loop that it is embedded in, see "Delayed Impact of Fair Machine Learning" https://arxiv.org/abs/1803.04383
The fact that you were hit is influenced by your risk factors. For example, maybe that is a bad intersection, and you always drive through it. Maybe it really is just the other driver being bad... but he and you happen to leave for work every day at a reliable time, making it likely that you will encounter him again. Maybe your car is hard to see. Determining "not at fault" is a fuzzy thing; perhaps a better driver would have somehow avoided the crash. The number of miles you drive is an influence on the number of crashes you get, and perhaps you drive more than is typical.
Actually, yes. In Russia we have a principle called DDD which can be translated as Give Way to a Moron ("Дай дорогу дураку").
People who avoid dangerous drivers, even when it hurts their ego, are less likely to get involved in a collision.
Also make sure the data you use is not a proxy for the above criteria
Part of the problem is that, when dealing with humans as the data set, the data is really messy and there's lots of noise correlated to historical circumstance.
I don't think that opening the algorithms is a sufficient condition but I think is necessary.
Maybe, but we know blatantly unfair algorithms when we see them. And we see them.
e.g. Remember the AI recruiting tool that automatically rejected women because that's what it learned from the input data?
https://www.theverge.com/2018/10/10/17958784/ai-recruiting-t...
They may happen to align, you cant force it though.
I agree that its the general case that matters. Impossible is a very strong word though.
How wide do you want to cast the 'general' net? You can still have limited scenarios that are useful. You don't have to model the entire universe.
[1]: https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...
Each are equal, 25% get 1200 on the sat, there is no allocative harm.
It is therefore fair, no?
If it is _literally impossible_ I must be misunderstanding something. Because I don't see the things he's mentioning as being mutually exclusive.
AI is just tools of humans. Any system that can be achieved with humans can also be achieved with AI.
These AIs take us backwards from our existing human systems in addition to being much harder to correct later.
The supermarket theft example in the slides you linked to is a prime example. In what store in America would that kind of display be deemed acceptable? Yeah, you save a few dollars a day on theft, but what kind of goodwill are you spending? Now that AI is here, people think they can get away with things like that because numbers.
I hope you meant it the other way around.
Don't misunderstand me; I'm not saying that discrimination doesn't exist or that we live in a society with no biases. I'm saying there is no valid reason why AI should take us backward in that regard.
Computers are extensions of the will of man. Our AI researchers and developers should ask themselves what kind of society they want to create. Whether they choose to turn a blind eye to discrimination and bias, or choose to actively combat it, then that will determine the result. Civil rights didn't just happen; there were people fighting for it actively. The same has to occur in the digital world if the same result is desired.
I'm going to repeat this again, since many people struggle with this.
It is _literally impossible_ to achieve the best utilitarian outcome, the most procedurally fair outcome, and a representatively fair outcome.
Anyone designing algorithms will have to make tradeoffs along this frontier.
For a far better argument for the above than I can make in a Hacker News thread, please see these slides from Chris Stucchio:
https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...