I'm really sympathetic to your point. I would not want the guilt or innocence of myself, a family member, a friend, etc depend on the numeracy and statistical literacy of a jury, with the stats explained to them through lawyers (even my own!)
As to the OP, the key sentence for me is:
> ...the fire had been started by a discarded cigarette, ... the other two explanations were even more implausible
Only TWO other possible explanations for the cause of a fire!?!?! Regardless of the word choice, I think the appeals court has a better intuitive understanding of some of the pitfalls of model-based Bayesian statistics than the OP (namely, if you accidentally put zero weight on parts of the prior, you will necessarily put zero weight on those parts of the posterior).
edit: if I'm going to be listing credentials, I should point out that my econ phd is in econometrics; I don't want to give the impression that my MS in stats is a sufficient credential to teach econ phd students.
How is this different then an expert on autopsies incorrectly does the job which leads to a potential false conclusion.
In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?
Even on HN you'll find people making simple mistakes.
Someone confidant presenting a mathematical argument would be persuasive to a jury, even if their numbers are totally bogus.
> In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?
People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling.
"We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."
>"We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."
Investigators have been known to be unduly influenced by pretty much any kind of evidence imaginable and then all further evidence is construed to support the original assumption/assertion. I need a more concrete difference or divider to promote other methods above statistics.
A different way of phrasing this is that investigators have been known to incorrectly apply even standard tools that they've been explicitly trained in.
> I need a more concrete difference or divider to promote other methods above statistics.
If by "promote" you mean that we should try to increase the statistical literacy of lawyers, police, judges, and the general public, than hells yeah. But encouraging armchair statisticians to try to determine causality and guilt from badly-collected observational data that's analyzed by financially-invested parties...? In the highest stakes setting we have? Really?
That is why both the defense and the prosecutor can bring forth their own expert witnesses.
How is this case different?
Additionally I'd guess most times if you make a significant error during an autopsy, you are aware of it. Whereas it's much more difficult to know whether you made an error in your statistical analysis or not.
That was impressions from remember articles where expertise in the above three fields was tested independently and experts were found to lack predictive power. I would have to search for that articles again if I wanted to provide a reference however I am working from memory.
Why should I trust them less for stats? Math does hold a honorary position in society while also being often being disliked(USA here) I will have to think about weather that is enough to single stats out from the others.
For non-experts in forensics vs statistics can an average jury member make better decisions about expert level results in forensics compared to statistics/math? It is possible that statistics and math are just less common knowledge and most jury members would then be able to discerne finner differences in forensics then compared to statistics. If this is the case though there is nothing incorrect about the tool, statistis which seems to be implied in the original article, and it is just a lack of common understand that in theory if correct would allow for a new tool to be used.
In that case, her chance of being present would be 1 in 1. But it doesn't mean she did it.
This is in an English court, and it is a civil case. Thus, there is no "conviction". For a criminal case the standard is "beyond reasonable doubt". For a civil case the standard is "balance of probability".
So, no-one is going to jail or getting a criminal record. But they might have to pay compensation to someone else for fire damage.
Such is the case in a fearful populace, and one that demands vengeance over justice. I appears as though something bad happened, so someone must pay and ... here's a magic forumla with missing variables that says you might have done it so ... GUILTY.
The case the parent was talking about took place in the Netherlands. I don't think they even use juries in the Netherlands (though I could be wrong about that).
Even in the UK -- the jurisdiction the article is about -- juries are purely random. We don't do the American practice of lawyers from both sides being able to reject jurors for their own reasons (as opposed to an actual good reason like their knowing the defendant, and the judge makes the decision on that).
http://www.dutchnews.nl/news/archives/2008/04/judge_calls_fo...
[0] https://en.wikipedia.org/wiki/File:Anglospeak%28800px%29.png
[1] https://en.wikipedia.org/wiki/File:LegalSystemsOfTheWorldMap...
[2] https://en.wikipedia.org/wiki/File:Democracy_Index_2011.png
As for the distribution of languages and laws, it's largely due to history. Civil/code jurisdictions show the extent of the former Roman empire and later the influence of Napoleonic Code. Common law countries are almost universally within the Anglosphere and have a common legal heritage commencing in the 1066 Norman Conquest.
The initial analysis used Fisher's Exact Test, which is a frequentist rather than a Bayesian method.
I'm not sure I understand what you suggest there, because, what else is there?
Let's say a murder conviction is based on 37 different security recordings clearly showing the defendant murdering the victim. The recordings make it a certainty that the defendant is guilty.
But what is "certainty"? It's an expression of probability, and that comes from statistics. Given the evidence, what are the odds that the defendant is not guilty? Extremely low. We have to come up with pretty crazy alternatives. The probability of the defendant being innocent in this case might be, let's say, 1 in 10^20.
We don't need to actually carry out this calculation, because we don't need the precise probability. It's clear that the probability is extremely low, so we can take a shortcut and refrain from figuring out exactly how low. But that shortcut is still ultimately an exercise in statistics.
Having something on a security recording can be classed as "hard evidence".
The case in question was one of using circumstantial evidence alone to "statistically" decide whether she was guilty or not.
The law differentiates between circumstantial evidence and hard evidence. People aren't supposed to be convicted on circumstantial evidence alone.
IANAL but I think this is very wrong.
Obviously the court will prefer strong forensic evidence if it can get it. But when it isn't, defendants can be, and are, convicted on circumstantial evidence, and this is not considered a miscarriage of justice.
If a security recording captures someone whose face can be clearly seen to be that of the defendant, and the recording shows them actually committing the crime, okay, that's pretty hard evidence. Anything short of that -- their face isn't shown clearly, perhaps, or the recording shows them being in the area but doesn't prove they actually committed the crime -- and you start having to talk about probabilities.
Even forensic evidence comes with probabilities attached. It's routine for fingerprint and DNA matches to be qualified with a probability of error. And often the true accuracy is overstated.
Circumstantial evidence is evidence which requires a logical inference in order to support the point for which it is introduced.
That is roughly how evidence is explained by judges to juries in courtrooms.
For example, the victim's blood on the defendant's shoe is noncircumstantial evidence because it directly supports the point that the defendant was present in the vicinity of the victim's body. However, finding at the crime scene a brown fiber which is same type of fiber used to stich the defendant's very expensive, limited edition gloves, would be circumstantial evidence because you have to logically infer that the fiber found comes from the defendant's gloves. Alternatively, finding gloves that are identical to the defendant's gloves is also circumstantial evidence unless there is other evidence showing conclusively that those particular gloves are the defendants, i.e., footage of the defendant leaving the gloves there or something.
The difference between the probabilities involved in the above inferences, and the analogous inferences ineluctably underlying any hard evidence, and those involved in circumstantial evidence like your coat fiber, are purely quantitative. In some sense, hard evidence is that which can be accepted "beyond a reasonable doubt", while circumstantial evidence is that which does not pass such a test.
Unfortunately, "beyond a reasonable doubt" escapes precise definition. Yet, courts need to function, so we tacitly agree that sufficiently large quantitative differences are (legally) indistinguishable from qualitative differences.
(EDIT: I'm saying the same thing as mikeash, whose response appeared while I was writing my own)
Noncircumstantial evidence directly supports the point for which it is introduced if the evidence is what it claims to be. The victim's blood on the defendant's shoe directly supports the point that the defendant was in the victim's presence when the victim was dying if it is established that the blood is the victim's blood.
Circumstantial evidence requires an additional inference beyond this step. The fibers are circumstantial because the fiber doesn't prove anything except that the someone wearing those gloves might have been at the crime scene at some point in time. You require the additional inference that the fiber was left at the scene at the time of the crime, which the fiber by itself does not prove but which can be "circumstantially" proved in combination with other evidence. For example, if the defendant, or some other witness, claims that the defendant was never at the scene of the crime at any point in time (specifically, but not limited to, periods prior to the crime), the presence of the rare fibers from his rare gloves is circumstantial evidence that the only time he was at the scene of the crime was when the crime occurred. In other words, the circumstances surrounding the fiber are the proof, not the fiber itself.
The (admittedly pedantic) point being made is that proving 'x' never directly supports anything other than the truth of 'x'. Moving from 'x' is true to 'y' is true always requires further inference.
However, the main point of this thread of discussion I think was that even things that legally are considered "direct evidence" - say, testimony of an witness to the crime - do nonetheless involve some inference...
Circumstantial evidence is indirect, like you being in the area of something, hard evidence is actually of you committing an alleged act.
The case in question was one of using circumstantial evidence alone to "statistically" decide whether she was guilty or not.
And yet, aren't all types of hard evidence really just probabilistic as well? For example, there is a non-zero probability that the murderer on the security camera is merely a person with an amazing physical similarity to the defendant. If we could develop an accurate mathematical model of human physical features, we could even estimate the probability that we're looking at a double and not the defendant.
Likewise, when a DNA match is used in court, the forensic scientist typically testifies to the probability of a false match. Once again, the evidence is probabilistic.
In both of these cases, the probability of a mistake is low. But it is non-zero, and it can be quanitified--if not with current technology, then at least in principle.
So, you will never get away from deciding cases probabilistically. And if you simply refuse to assign numerical probabilities to anything even when the circumstances readily permit it, you're not increasing your chance of being right, you're just sticking your head in the sand.
This is the problem with this approach. Let's say the guilty party is someone who looks like the defendant. Now it doesn't matter how many security cameras there are, they all record what happened: Someone who looks like the defendant commits a murder on camera. The number of recordings doesn't inherently show whether it was the defendant or a double.
Or let's suppose that the evidence has been fabricated by someone trying to frame the defendant. Someone who fabricates one recording to the satisfaction of the jury can generally fabricate 37.
Or perhaps it was the defendant on the video but it wasn't a murder, it was a staged public death in cooperation with the alleged victim who subsequently went into hiding.
All of these things are individually relatively unlikely, but they don't become any more unlikely as a result of additional recordings.
This is one of the reasons why conviction rates are generally so high. Suppose (picking a number out of a hat, since we can't really know) that 60% of the accused who go to trial are actually guilty as a factual matter. But of the ones who aren't guilty, 0.6% are because a double committed the crime, 0.5% are cases where the defendant is being framed by a third party or the police, 0.7% are cases where the defendant could show a lack of guilt but doing so would endanger someone else and the defendant prefers to risk prison over endangering that person, etc., and when you add up all the individually low probabilities it turns out that at least one of them has happened in a quite substantial percentage of cases.
The trouble is that the defendant who goes to trial and claims to have been framed is not believed, because only 0.5% of cases are genuine instances of framed defendants (and juries intuitively understand that), and the prosecutor is too often allowed to infer from this that the remaining 99.5% of defendants are actually guilty, rather than that some quite large percentage of them are not guilty but for some other reasons, each of which seems about equally unlikely in any given case and is thus subject to the same misleading inference. But the alternative, to rigorously consider each of the individually not very likely alternatives, is extraordinarily expensive -- which is why statistics is given the short shrift.
At the end of the day the problem is that proving guilt to a level of certainty that would satisfy us if we were being rigorous about it would also cost too much and be too difficult for prosecutors to prove in most cases to satisfy our political desire to incarcerate "bad people" on a mass scale without breaking the bank any more than we already do. Rigorous, large scale, feasible budget. Pick two.
The fact is there's no hard line between good and bad evidence, though I sincerely hope Bayesian equations are never the deciding factor in court cases. Math is great, but "probably guilty" just isn't good enough unless it's incredibly probable. As in, 99% probability seems to be a near certainty, but 75%? 75% is still a strong induction, but that's a 1/4 chance you're convicting an innocent person. So where do you draw the line?
Also remember that Bayesian Probability is still susceptible to bias extremely easy just by determination of how to define the variables. If I were looking to convince a jury, it would be fairly easy to tweak the variables and have a large degree of control over the resulting probability while appearing unbiased (because of Math!). So in this sense, I think that bayesian probability is pseudomathematics. Nobody's questioning the math, but the conclusions are not purely derived from math.
I don't understand what you're saying. The fact that being caught on camera is an inductive argument for guilt is ultimately based on statistics or probability. What is the probability that the person on camera is the defendant?
Where do you draw the line? The American legal system, for criminal cases, draws it at "beyond a reasonable doubt", and provides no numerical guidance for what that means. But there is a line, and not wanting there to be one doesn't make it go away.
If the court argued from human unreliability in enumerating options, the judgement would be sensible. But simply arguing from a mistaken understanding of probability makes it look silly.
So many convictions come based on simple circumstantial evidence without even considering the statistical basis for those circumstances. Many convictions are entirely emotional bias on the part of the jury. Why should allowing statistical analysis to be considered as evidence be considered worse than the status quo?
There never was solid evidence against her. She was a suspect because nearly everytime someone died in the hospital she was on her shift.
The initial chance of her being present when these supposed murders happened, and being innocent, was estimated at 1 in 7 billion. This number caused the police to focus their research completely on Lucia de B.
Eventually she was convicted for the murders based not on hard evidence or witnesses, but on statistics. The chance of innocence was put on 1 in 342 million.
The econometrist Aart de Vos was the first to notice that the initial Bayesian analysis was plain wrong. For example they had presumed the murderer had to be found amongst the nurses, other possibilities were neglected. They also hadn't corrected for combined P-values. He reduced the chance Lucia was innocent to 1 in a million.
The court said they had abandoned the statistical "proof", but remained of the view that it couldn't be a coincidence. Possible murder cases were chosen when they were negative for Lucia, other possible murder cases were left out of the equation. This further reduced the chance to 1 in 50.
Statisticians Richard Gill and Piet Groeneboom further reduced the chance to 1 in 9.
In my opinion statistics alone can never be adequate for conviction. It can muddy research and lead to confirmation bias. And the difference between a Bayesian chance of 1 in 7 billion and 1 in 9 is so big as to doubt the initial use of statistics even further.
http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_argum...