> The reality is humans are far worse
Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws.
An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way.
If you give a machine learning model a dataset where one feature appears entirely indicative (remember: correlation is not causation), it can overfit to that, even if that does not reflect reality.
I highly recommend reading "How big data is unfair: understanding unintended sources of unfairness in data driven decision making"[1], by Moritz Hardt, a Google machine learning researcher who has published on the topic (see: Fairness, Accountability, Transparency). It is a non-technical and general introduction to some of the many issues that can result in bias and prejudice in machine learning models. To summarize, "machine learning is not, by default, fair or just in any meaningful way".
Algorithms and machine learning models _can_ be biased, for many reasons. Without proper analysis, we don't know whether it's a good or bad model, full stop.
[1]: https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de...
516 is more than enough to fit a simple model. As long as you use cross validation and hold out tests to make sure you aren't fitting. 516 data points is more than a person needs to see to be called an "expert". Many of the algorithms I referenced used fewer data points, or even totally unoptimized weights, and still beat human experts.
>An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way.
We have ground truth though. Whether someone will be convicted is a fairly objective measure. Even if it's slightly biased, it's still the best possible indicator we have of whether or not someone should be released. If you had a time machine that could go into the future and see who would be convicted, would you still argue against using that information, because it might be biased? Leaving people to rot in prison, even if all the statistics point to them being very low risk, is just wrong.
>"machine learning is not, by default, fair or just in any meaningful way"
Humans are not, by default, fair or just in any meaningful way. Nor are they accurate at prediction. Any argument you can possibly use against algorithms applies even more to humans. That's my entire point. You should trust humans far, far less than you do.
Which is why I don't trust the people picking the algorithm. You still have human bias, but now they are easier to hide behind complicated algorithms and unreliable data.
edit: removed original editing error
edit2: You say I should trust the algorithm, but y9u seem to be going out of your way to ignore that the algorithm itself has to be created by someone. You haven't reduced the amount of bias; trusting an algorithm simply codifies the author's bias.
Look it's easy to verify whether parole officers are better at predicting recidivism than an algorithm. If the algorithm is objectively better than it should be used.
Given that it's easy to construct an example of how you could deliberately do this, and it's so easy to accidentally overfit machine-learning algorithms, we should be very concerned about people accidentally doing this. An easy way would be to try a few thousand different algorithm variants and have a biased group of people eyeball the results to see which ones look good. If those people are racist, for example, they could subconsciously give undue weight to type 1 errors for white prisoners and type 2 errors for black prisoners, or vice versa.
The outcome of the process would be an algorithm that is "objectively better" by whatever measure you optimize it for, but still unjustly worsens the situation for some group of people.
A potential advantage of algorithms and other rules is that, unlike the brain of the judge, they can be publicly analyzed, and the analyses debated. This is the basis for the idea of "rule of law". Aside from historical precedents, though, the exploitation of the publicly-analyzed DAO algorithms should give us pause on this count.
Deeper rule of law may help, but it could easily make the situation worse. We must be skeptical, and we must do better.
Out of curiosity, if you find that in 240 out of 261 small round blue cell tumors, your algorithm can make a better treatment decision than the pathologist, do you wait for more, or get the algorithm into trials ASAP?
Careful now. I didn't mention how long you might be waiting. This is a domain specific trick question.
Furthermore, we have an appeals process to address sentencing mistakes. It isn't easy, but if the risk scoring algorithm turns out to be materially wrong and contributes to a harsher sentence than warranted, that can be considered a factual error and the basis for resentencing.
It's almost as if our justice system is geared towards a reinforcement learning approach, due to the consequences of both action and inaction...
Sometimes there is a minority report.
It's the same reason many people oppose electronic voting. Sure, letting humans count votes seems inefficient, and might enable some small scale fraud, but it's much more difficult to commit large scale fraud if there are many people involved. When you use voting machines, you put a tremendous amount of power into the hands of the people who design and build those machines.
Those algorithms are also designed by humans and codify biases and prejudices and injustices that are considered OK by society into them.
Besides, it's like going for a run. Other might might be far better at it, but we want to do it ourselves, even badly, than just be spectators in our own life. Same for humans becoming spectators while algorithms run business "better".
One quality that's not accounted for in accessing better is "more participative".
And I don't think preferring to do stuff yourself is an excuse. These are people's lives on the line! More accurate predictions mean fewer people in prison, fewer murders and robberies, etc. Or in a medical domain where more accurate predictions mean fewer people die of terrible diseases. You should always prefer algorithms if there are consequences for worse predictions!
But the humans would have to explicitly build the biases into the algorithm. Many human biases are unconscious, so we aren't aware that our conclusions are biased by race, a subject's attractiveness, etc. Going with an algorithm would minimize unconscious biases, and prevent us from reaching a conclusion based on emotion and then just back-fitting a rational-sounding argument.
Explicitly as in coding them, but not necessarily as doing it consciously. After all biases are just how people see reality, so it would be natural to code based on one's biases.
Also it's less the biases (that are inevitable), and more the blind following of biases (algorithmically) that I'm concerned about.
>Going with an algorithm would minimize unconscious biases, and prevent us from reaching a conclusion based on emotion
The latter is not necessarily (or even commonly) bad, it's more of a citation needed kind of thing. I'd rather we DO reach conclusions based on emotions. Naked logic sees everything as black and white, it's emotions that make civilisation nuanced (including empathy, pity, gut instinct, abhorrence, etc).
Can you do the same with the human subconscious? Maybe, given lots of time and effort.
How do you evaluate success? Can you do it at scale? I don't think so.
And knives can be used both for murdering people and spreading jam on your bread.
The answer to "racist emotions" are emotions of empathy and love towards the other, and not some objective logic that one can follow to fix racism.
After all, one could always use objective logic to justify racism if one wants to -- e.g. "blacks are involved in a higher percentage of violent crimes compared to their population". That's a totally objective statement, it just skips trying to find the reasons why that might be so, and uses the numbers to justify thinking of them as violent inherently.
How can you be sure your 'fair' algorithm is running for each individual court case? That it hasn't been tampered with, swapped out, hacked or otherwise changed? Do you give the entire court room the ability to look at the machine and what it's running?
How do you then stop any of them messing around with it when you're not looking?
You've then got the unfortunate implications that an automated system might create. Okay, the court system very much isn't neutral in regards to race, wealth, gender, nationality, religion or anything else. How do you stop an automated system discriminating based on those factors? Because there are probably different statistics and likelyhoods of different groups committing crimes.
And there are various other issues too. Do you trust a small group of people, a company, some open source contributors or an individual to program a system to make 'fairer' decisions than a country's worth of judges? Because the amount of factors that go into a court case means that a piece of software has to deal with a thousands or even millions of possibilities. Which provide a place for an unscrupulous, lazy or shortsighted programmer to screw things up.
And there's also the old trust issue. People tend to trust other people more than machines, especially those seemingly associated with a faceless corporation, large government or unknown force. A system that replaces people, no matter how good the algorithm, would get a lot of bad press in this sort of situation. People won't be happy about being sentenced to prison by a computer program. Heck, even a judge using a smartphone app would cause an uproar about how 'soulless technology is replacing human intuition' or something similar.
As for trying to make human judges blind to race and gender. That's its own challenge. Facial expressions and courtroom behaviour can indicate whether someone is guilty or not about as much as what they actually say.
Try and anonymise that, and you end up in a situation where those who can act regretful over the phone/without being soon have an advantage over those that can't.
There are also situations where being blind to race and gender would be a bad thing.
Economic information can cause bias, but it can also be used to treat a defendant more fairly as well. If someone was really poor/starving on the streets and stole to stay alive, shouldn't that be indicated somehow? There's a difference between someone like that and a rich guy who stole because he was bored or wanted more.
But in the worst case, you can print the algorithm out on paper. They are generally simple linear models or decision trees. You could easily verify the output by hand, albeit a bit tediously.
In the case of more complicated models like random forests or ensembles, you can generate lots of artificial data, and then train a simpler model to approximate its output as best as possible. Then you can print out that model.
In the old days, they printed out actuarial tables. Big tables that you would look up your case, and find the recommended output.
>Try and anonymise that, and you end up in a situation where those who can act regretful over the phone/without being soon have an advantage over those that can't.
That's exactly the problem. People vastly overestimate how well they can read facial expressions. I recall studies that officers couldn't tell liars from nonliars better than chance, but they believed they could.
The attraction of an algorithm is that you can control strictly what information goes in, and what objective is targeted. A judge may - consciously or not - use the information of the accused's skin color. A judge may - again consciously or not - target objectives besides lowering recidivism.
An algorithm isn't always better, but when it produces unwanted results, we can criticize it, think about why it went wrong, improve it. When a judge makes mistakes, there's not any easy way to improve him.
I think artificial objectivity is "bigger" than artificial intelligence. It's not that ML algorithms are so much better than human brains at what they do - it's that we can have more confidence about how it does it. Oddly enough, considering how they're often criticized for opacity.
I believe that now that there is much fuzz about AI you can improve lots of things. I would like to see in 200 years if you can - no, you can't, why? For the same reason you hardly can change things now: "the algorithm has been studied, improved, tested over and over" -> this is the law. But who says it's correct or fair? A judge, exactly because he is a human, can improve. How? For example, with a feedback system or by requiring explanations for his judgement (based on facts, not moods) - if he doesn't do so, he shall be accountable for his actions.
I am not too much into law or politics, but things are that way since always and hardly have changed. But we know how things will go. Algorithms/machines will slowly replace everything, because they lower the costs with the promise that things will be more dynamic and that life will improve - however, the usual gang of people will manage to be "more equal" than others, also when the algorithm works perfectly.
> The attraction of an algorithm is that you can control strictly what information goes in [...]
How is this attractive? Information can be easily manipulated, delayed, withheld. Essentially, you are just shifting the accountability - from the judge to the guy who feeds the algorithm.
And by the way, we are talking about judges who don't apply the law appropriately. Let's talk about those judges who applying the law put their lives at risk - or simply get killed. Shall we? Can we at least trust those judges?
If human judges do better at this task, it's very easy to test it. We ask them to predict what criminals are low risk or high risk. And they barely do better than chance, and are easily beaten by the simplest statistical baseline. Yes algorithms have edge cases, but humans have far more.
But I believe an even bigger advantage of algorithms is repeatability. Yes initially the algorithms might be biased, but they will always given the same input, deliver the same verdict. And over time we will be able to perfect the algorithms.
It's like automated testing. Initially at first automated testing is far less effective than manual testing, but as you add and improve your tests they outperform manual testing almost always.
And just like with testing you reduce the need for manual testing, algorithms like these do not have to completely eliminate human judges, but can reduce the amount of cases they are needed for, with improvements in sentencing and processing times.
In my opinion, such areas definitely needs some degree of checks and balances to prevent power abuse. Reducing risk assessment to a ML style single algorithmic number would give the people in charge of the algorithm an awful lot of power.
I do agree that if humans were entirely moral and ethical, ML can be a huge net gain. I'm not sure I trust humanity that much, personally.
Right now many interventions are little more than guesses that may actually be making things worse. It's a ridiculous state of affairs for 2016.
Right, so who makes the algorithms?
The algorithm aversion paper is here: http://opim.wharton.upenn.edu/risk/library/WPAF201410-Algort...
The attraction bias is found here: http://lesswrong.com/lw/lj/the_halo_effect/
The hungry judge study is found here: http://www.scientificamerican.com/article/lunchtime-leniency...
That algorithms almost always do better than humans is backed up by this: http://lesswrong.com/lw/3gv/statistical_prediction_rules_out... and https://meehl.dl.umn.edu/sites/g/files/pua1696/f/167grovemee... which also mentions the 1928 study.
I agree that human judgement is often flawed and biased. This particular algorithm appears to be always racially biased[1], and wasn't even tesed for it. Substituting flawed human judgement for systematic bias is a bad, bad idea.
[1] https://www.propublica.org/article/machine-bias-risk-assessm...
Right, but there is plenty of evidence that this (the statistically worse criminal records) is an artefact of sentencing[1].
Using that biased feature as an input is going to make it worse: the algorithm will recommend higher sentences because previously blacks had previously had more severe sentences.
This is almost the exact definition of implicit bias[2]
[1] eg http://www.wsj.com/articles/SB100014241278873244320045783044... but plenty of other examples.
Second, a criminal record is the crimes you committed in the past, not the sentence a judge gave. The algorithm would see your record and determine your level of risk, along with other objective factors.
The algorithm has no bias. It sees the data and uses it to make the most accurate predictions possible. It doesn't care about race, and it is blind to it. A white criminal with the same record, should be given the same prediction as a similar black criminal. And the predictions it makes should be pretty accurate and objective.
The reality is humans are far worse. We are biased by all sorts of things. Unattractive people were found to get twice as long sentences as attractive ones. Judges were found to give much harsher sentences right before lunch time, when they were hungry. Doing interviews was found to decrease the performance of human judges, in domains like hiring and determining parolle. As opposed to just looking at the facts.
Even very simple statistical algorithms far outperform humans in almost every domain. As early as 1928, a simple statistical rule predicted recidivism better than prison psychologists. They predict the success of college students, job applicants, outcomes of medical treatment, etc, far better than human experts. Human experts never even beat the most basic statistical baseline.
You should never ever trust human judges. They are neither fair nor accurate. In such an important domain as this, where better predictions reduce the time people spend in prison and crime, there is no excuse not to use them. Anything that gets low risk people out of prison is good.
I believe that any rules that apply to algorithms should apply to humans too. We are algorithms too after all. If algorithms have to be blind to race and gender, so should human judges. If economic information is bad to use, humans should be blind to it also. If we have a right to see why an algorithm made a decision the way it did, we should be able to inspect human brains to. Perhaps put judges and parolle officers in an MRI.