Machine learning is going to be very good at identifying proxies for features we don't like, even if we don't directly provide those features. Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc.
If we want machine learning to ignore these kind of things it's probably going to have to be a separately weighted training goal that racial groups and so on need to have equal outcomes in the final results.
That's really what AOC is going for here. The black box will happily learn racist ways to weight things all on its own, we can't just throw up our hands and say oh well, if the algorithm is denying black people for home loans then we just have to accept it.
Because, in the short term, being racist is an effective greedy strategy. It's not your business's problem that, say, black people have higher rates of default (or whatever, example). But it also leads to bad social outcomes in the aggregate, and we've agreed that shouldn't be a factor in home loans. And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics.
Off-topic-ish, but only about 10% of Irish people have red hair.
This is not wrong, but it boils down to "don't make mistakes when optimizing functions". It's offensive because it's coming from someone who has nothing to contribute except "do your job better". It's like saying we have a problem with planes crashing, so engineers have a moral duty to build better planes. As though they weren't trying to build planes that stay in the air already. You can make serious sounding statements, or you can impose fines, or you can randomly pick people to publicly shame, but none of those things are positive contributions to hard problems.
Mechanical engineers are taught from the beginning of their careers to think carefully about the consequences of their decisions.
Computer engineers, not so much. We're taught how to optimize, and make machines that optimize, and make optimizers that optimize our optimizing machines, but it's pretty rare to encounter a data ethics course, and they're almost never program requirements.
Facebook got in trouble for algorithmic redlining in 2013, serving loan sharky ads to black people (which its algorithms learned via proxy features, like zip code). Does Facebook hire third-rate engineers who would point the engine on a plane the wrong way? No! Facebook has some of the best people in the industry! But even the best people in the industry were and in many cases still are not fully considering the consequences of our designs, because we weren't trained to.
Wrong. We had two courses at my school. From the ABET Criteria for Accrediting Engineering Technology Programs: "(e)Include topics related to professional and ethical responsibilities".
The issue here is proxy features, and that it isnt as obvious if your statistical model is doing something wrong. You identified zip code and race. What about number of cats? Would you have a particular gender and marital status in mind if a given data sample had a large value in the numberOfCats column?
So you ban making decisions based on number of cats. But amountOfKittyLitterPurchased, numberOfCatPicturesViewed, numberOfDislikesOnDogPictures, etc could be used as a proxy for numberOfCats. Try to come up with an exhaustive list of all features which could be used as a proxy for some protected class. Starting with politically incorrect jokes will get you part way there, assuming you possess an exhaustive list of all politically incorrect jokes. But you will need to be pretty creative to come up with more.
https://www.abet.org/accreditation/accreditation-criteria/cr...
As a software person, I have absolutely no issue with telling other software people to do their job better, and I see no shortage of people that aren't even trying on that front.
Its your comparison between a process that takes safety, reliability and a reduction of harm as a core, first value, and a process that cares first about money, and second about novelty and somewhere way down the line people, maybe, that is offensive frankly.
If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the first place.
Most people here are male, were once (or are) under 25 and likely drive. How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? Quadruply so if there are prior citations/accidents?
Shit, with a spotless driving record, my own insurance went up when someone t-boned me. Other driver was at-fault and I subrogated their insurance; didn't claim with mine. But I'm being lumped into a higher-risk pool with my own insurer for factors completely beyond my control-- because to some algorithm, being in an accident means I'm more likely to be in accidents, and all drivers who are involved in collisions are higher risks than drivers who aren't. Note how the fact that I wasn't at fault isn't factored in. This makes sense to you?
The social training data in much of Europe is skewed towards hating Gypsies and anyone perceived to be one. But it's the same as the ex-con problem-- deny them jobs, beat them, chase them away, and of course an ex-con or young Romani will commit future crimes. You're shaping behavior to validate your training data instead of making unbiased ("fair") predictions. It's the equivalent of betting on a horse, injuring its competition, then patting yourself on the back for being so goddamn good at this game.
You can't look at the world in terms of sheer numbers. The numbers themselves are dishonest.
isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.
Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident.
The latter is more likely to be wealthy enough to be able to pay cash for repairs and keep the entire incident off the radar. No insurance claim, no citations, no evidence. The 18-year old has no choice but to make an insurance claim, go to court and take points on their license.
Can we truly conclude that 18-year olds with red Mustangs are worse drivers than their elder counterparts?
(And yes, I realize that in the case of insurance, it's in their interest to minimize claims so in that context this is a valid conclusion. But this same skewed data gets shared outside of the industry as well.)
this is not a fair example; by assumption we have much more information about the 58yo than the 18yo. remove the bit about the midlife crisis and there's no reason to think the 58yo is anywhere near the same risk to insure. as a group, young people are less experienced drivers, have less facility for judgement, and the males probably drive more aggressively.
unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this.
and by the way, I'm a young male who drives a fast car.
I don't disagree. In this context, if older men make fewer insurance claims, they're better customers. On paper, he's less of a risk (to the insurer). In reality, they're both risks to other drivers. But only the 18-year old will be held accountable.
But assuming the under-25-and-male crowd are the worst drivers on the road based on number of insurance claims is not necessarily true, yet it's the basis for this oft-repeated assertion. That's my point. What isn't the insurance industry's data telling us when we use their data to make these conclusions? To get more accurate numbers, you need to be looking at the demographics of all body shop customers and the nature of their repairs. What demographics get the most front-end work done that isn't covered by an insurance check?
This new age of computing seems to be to make machines understand how the world works by putting blind faith in statistics, without ever stopping to question where the stats came from or how accurate they really are. The numbers aren't handed to us on golden tablets carved by God himself-- we get them from imperfect systems designed by imperfect humans. And then we teach dynamic algorithms how reality works based on this data and sell them as solutions, which we trust absolutely.
> and by the way, I'm a young male who drives a fast car.
No offense intended. I once was, too.
if you are just using it as an example of a misleading dataset, we are probably in agreement.
a better argument would be something like: suppose we detect a gene that indicates probability 1.0 of disease that causes the person's left arm to fall off.
it's then not a matter of "insurance" but a matter of "are we as the pool willing and able to pay for this?" recall company needs some profit as well as cost of coverage to maintain overhead so it's a different argument now.
And changing the algorithms isn't the right way to fix those external factors. If you try, their self-reinforcing nature is going to cause exactly the opposite of what you want.
To stay with the crime vs. race example, assume that the ratio of black to white criminals is greater than you'd expect from the ratio of blacks to whites in the overall population, and that is due to self-reinforcing external factors. Your algorithm for deciding whether to parole a prisoner picks up on that statistical regularity and rates blacks riskier than whites. Because you want to achieve representational fairness, you make the algorithm race-blind to equalize the rates.
What happens? The black prison population shrinks. But your algorithm wasn't wrong when it predicted higher recidivism risk, so the black crime rate rises. Your algorithm now requires an even stronger correction to remain race-blind. Black criminals realize that they have an easier time getting paroled, so the black crime rate rises some more. Debiasing the algorithm has made the situation worse.
In general, when you find out that your machine-learning model has some undesirable bias, the correct response is not to blind the algorithm. Instead, you should fix the data-generation process (the real world) until your model no longer picks up on the bias.
Edit: For a mathematically precise description of how attempts to make an algorithm more fair cause worse results when you consider the feedback loop that it is embedded in, see "Delayed Impact of Fair Machine Learning" https://arxiv.org/abs/1803.04383
The fact that you were hit is influenced by your risk factors. For example, maybe that is a bad intersection, and you always drive through it. Maybe it really is just the other driver being bad... but he and you happen to leave for work every day at a reliable time, making it likely that you will encounter him again. Maybe your car is hard to see. Determining "not at fault" is a fuzzy thing; perhaps a better driver would have somehow avoided the crash. The number of miles you drive is an influence on the number of crashes you get, and perhaps you drive more than is typical.
Actually, yes. In Russia we have a principle called DDD which can be translated as Give Way to a Moron ("Дай дорогу дураку").
People who avoid dangerous drivers, even when it hurts their ego, are less likely to get involved in a collision.
Also make sure the data you use is not a proxy for the above criteria
Part of the problem is that, when dealing with humans as the data set, the data is really messy and there's lots of noise correlated to historical circumstance.
I don't think that opening the algorithms is a sufficient condition but I think is necessary.
Maybe, but we know blatantly unfair algorithms when we see them. And we see them.
e.g. Remember the AI recruiting tool that automatically rejected women because that's what it learned from the input data?
https://www.theverge.com/2018/10/10/17958784/ai-recruiting-t...
Maybe???? But it seems very dishonest.
I think, if anything, the models are probably too true for our tastes as social creatures who evolved to avoid social conflict (being 100% honest with your unruly neighbors can lead to a loss of harmony that is much more damaging than just putting up with some disturbances). Our idea of "unbiasing" the models is more likely actually making them biased in a way that pleases our monkey brains.