And jurors.

This one's my favourite:

The jurors who used a Ouija board to find a murderer guilty - https://www.abc.net.au/news/2018-05-08/ouija-board-juror-mis...

The Drunkards Walk talks about the misuse of statistics in court as well. Not only do they get the math wrong easily, or make common mistakes ("the odds that it's raining given the ground is wet, is the same as the odds that the ground is wet given it's raining"), but if someone accurately states there is only a 1 in 100,000 chance of something happening by chance, the jury will think that means the "something" was not by chance, instead of "oh, so it happens by chance to thousands and thousands of people" like they should.

Even forensic science can be used questionably. Really, once you're in the court room it's entirely rhetoric and fudge-factor.

My favorite case was where someone was accused of arson because the wood floor was found to contain traces of gasoline, even though none was found on the carpet or padding on top. Turns out that further testing by the defense showed lead also. Which means that the gasoline came from a cheap contractor several decades ago that used (at the time leaded) gasoline as a lacquer thinner for the hardwood floor. And the burn patterns on the floor which indicated an accelerant, were due to wear patterns on the carpet above which allowed more heat to penetrate.

> Really, once you're in the court room it's entirely rhetoric and fudge-factor

I agree with all the skepticism of particular types of evidence in this thread including witness testimony and forensic evidence (I'd also add even confessions to this list). But this rather extreme conclusion doesn't follow. Evidence can be uncertain and hard to interpret, but still lead us to truth. You just need more than one data point.

In a typical criminal trial (assuming the defendant is guilty) the prosecution generally will not hang on a single dubious piece of evidence. They will often assemble a veritable mountain, if they can, because they know that a competent defense attorney will make sure that the jury knows better than anyone the various ways in which evidence can mislead.

This is not to say that everything is rosy in the criminal justice system, and that juries always get it right. There are lots of big problems: Juries are often mysterious and unreliable. Defense attorneys often do a bad job (meaning, among other things, that the prosecution can get sometimes get away with presenting a weak case based on shoddy evidence). And, lest we forget, huge numbers of criminal prosecutions end in a guilty plea, and never go to trial in the first place (sometimes for good reasons, sometimes for bad).

But I want to push back on the extreme notion that the outcome of every trial is all about rhetoric. Sure, it plays a role and some cases are better than others. But I've never seen one that was "entirely rhetoric and fudge-factor." (I'm sure it happens, but not often enough that I've ever seen it.)

"Forensic science" has a long history of being flagrant psuedo-science, very often being a morally bankrupt abuse of statistics.

My favorite example:

>In one case, West claimed to have matched the bite marks in a half-eaten bologna sandwich to the defendant. The jury convicted. (The conviction was overturned on appeal when defense lawyers discovered that the autopsy report recorded a partial bologna sandwich in the stomach of the victim.)

https://www.washingtonpost.com/news/the-watch/wp/2015/02/17/...

Agreed.

Pretty much everything in forensic science turns out to be junk science over a long enough timeline. Hair, bite marks, fire progression, all junk. There's even a few cracks in the fingerprint wall. Don't get me started on all the field test kits of various types that LEOs use.

The only things that seem to be reliable are things that don't have their roots in forensic science (e.g. using DNA to identify people) and even then you still have to depend on a crime lab (run by the people doing the prosecuting) to not be sloppy.

>crime lab (run by the people doing the prosecuting) to not be sloppy

Oh yeah. Not just incompetence, but actual malice. Recently in Massachusetts there were two high profile state forensic drug lab scandals: One where a chemist was functionally incapacitated by taking all the drugs she should have been testing [0], and one where the chemist was falsifying positive tests (!!!!!!) [1]. More recently, the state police are known to be falsifying overtime records [2]. I can't even begin to imagine what it's like in "stereotyped as corrupt" states.

[0] https://www.cbsnews.com/news/6000-drug-cases-linked-to-rogue...

[1] https://www.cbsnews.com/news/more-than-21000-drug-conviction...

[2] https://www.masslive.com/expo/news/erry-2018/12/1fc00a248688...

> and even then you still have to depend on a crime lab (run by the people doing the prosecuting) to not be sloppy.

Which CSI has done a wonderful job of turning into super-cops in the eyes of jurys.

This has had the opposite effect of what you seem to be implying. It's something that's come up quite frequently in discussions of jury nullification [1]. Jury nullification being when a jury ends up voting not guilty, even when they believe the defendant guilty, generally because they do not find the law (or possibly the punishment) just.

For decades a rate of about 5% for hung juries was typical. In more recent years some jurisdictions have seen more than 20% of cases end in hung juries. [2] Some have seen this as evidence of a rise in jury nullification, but others have argued that it's due to the 'CSI Effect' [3] with these individuals believing that, because of shows like CSI, the standard of proof required for a conviction seems to be rising - along with a reduced weighting given to things like circumstantial evidence.

[1] - https://en.wikipedia.org/wiki/Jury_nullification

[2] - https://www.washingtonpost.com/wp-srv/national/jury080299.ht...

[3] - https://en.wikipedia.org/wiki/CSI_effect

That sounds like a good thing to me to be honest

The article about the CSI effect says it's believed to work both ways. Jurors expect more forensic evidence but put more faith in it.

High quality video can be quite useful for getting closer to the truth in many cases. Fits under your tech not developed for forensic science category though.

While we're at it, most of "ballistic fingerprinting", as well.

And along those lines, I've heard that DNA tests aren't reliable either.

The problem with the DNA tests we use today, assuming even perfect cases with zero contamination or mistakes (whole other argument there), is that (in my non-expert understanding):

    * Taking the samples and using several solutions to snip them at given patterns.
    * Taking another solution and promoting replication of the fragments.
    * Using some dye and a weight sorting channel to bin the fragments by weight.

This results in a kind of 'bar code' that graphs the distribution of weights as a VERY crude hash of samples of DNA.

It's pretty useful for determining things like IF it is LIKELY that individuals are related.

It's also pretty good at confirming negatives (We're sure someone is NOT related to X).

It's not that great at confirming positives. That is, the results are both subjective and ambiguous given the hashing. A "positive" result here is really more of an "OK, it's likely we should run the real and expensive check, evaluate if someone might be a suspect by other merits, etc."

I would, offhand, consider a "positive" above to be enough evidence to produce /suspicion/ and /warrants/ to locate other specific evidence to ascertain an actual guilt or innocence based on harder evidence. If an actual "sequence the whole set of samples" option enters the realm of feasible tests then it would also warrant actually doing that.

Come to think of it, I'm not sure how the Ancestry/etc novelty DNA tests work. I'd assume they've isolated a few specific markers they're looking for and the processes are optimized towards identifying those and comparing combinations for those specific traits.

You’ve described electrophoresis DNA fingerprinting correctly, then jumped to the unjustified conclusion that it’s a VERY crude hash.

Not so, it’s a fairly excellent hash, one with few collisions. Better than a fingerprint. If it looks like your DNA, it most likely is.

Source: was an electrophoresis tech (not forensic) in a previous life.

And yes, the usual genetics sites do SNP tests, which are cheaper than full sequencing but mean that they only find what they’re already looking for.

The price of full sequencing is dropping rapidly, I look forward to it being a standard part of medical practice, it will save a lot of lives and improve quality of care.

Better than a fingerprint is still not great!

Well, several decimal orders of magnitude better than a fingerprint, to be accurate.

DNA tests prove a very specific narrow fact, and it’s up to lawyers to string together a hypothesis in order for it to have relevance.

Unfortunately all too often the DNA test result is used as proof of a hypothesis, not of the mere presence of surprisingly similar DNA in a swab sample.

Apropos of this, but about human incompetency rather than DNA tests, the Phantom of Heilbronn. A serial killer so well concealed that her very existence was only given away by DNA traces.

https://en.wikipedia.org/wiki/Phantom_of_Heilbronn

Your Wikipedia link says that the Phantom is a myth and the DNA was contamination from the kit factory.

Yes it does, good reading skills. That's the point. Human incompetency, related to DNA forensics.

I didn't outright say that, though; the idea was that I would prepare you for a story of a serial killer so well concealed that her existence could only be inferred from trace DNA, but THEN there's the surprise twist - there is no serial killer, it's a trick of human incompetency. BOOM!

The twist has a much greater effect because I didn't give it away at the start, but the twist also ties back in to my original statement about human incompetency - but NOT where you may have been expecting, and subverted expectations are a key part of surprise twists - and the story is also about DNA forensics. That's the previous subject again, the reader having been led around in an interesting and surprising circle.

There is a lot of sketchy evidentiary techniques that get in, for sure. That said, the vast majority of prosecutions are based on extremely strong evidence. Reading the news gives a misimpression because you only see the edge cases. But for every “making of a murderer” there are ten guys who are convicted based in being caught on security cameras, and credit card reciepts tying them to the scene of the crime, and witness testimony.

I once had to go to court as an individual had attempted to stab myself and another (we had tried to stop them from escaping the police).

During the suspect identification process I failed completely; the suspect was bald and had a spider tattoo on their face. I realised that I couldn’t even remember how they looked, it shocked me quite a lot. I assume it was the adrenaline.