If field drug tests were actually accurate, they wouldn't be bought and used. Similarly, if an expert witness fails to give prosecutors the results and testimony that they want, they wouldn't be hired again.
[1] "How a $2 Roadside Drug Test Sends Innocent People to Jail": https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside...
In case any of you are interested, or completely flabbergasted as I was, by the idea that law enforcement's purpose and raison d'etre could ever become as distorted and contorted as this comment mentions, you should definitely read more on Walter "Johnny D." Macmillan [1] or watch the movie based on his story: Just Mercy[2].
Absolutely mind-blowing stuff.
The guilt I feel when I introduce a software bug that effects my corporate customers can sometimes burn my motivation for WEEKS. Who are these people?!
Exhibit A: Annie Dookhan of the Massachusetts State Crime Lab who is suspected of falsifying thousands of drug tests
[0]https://www.cbsnews.com/news/massachusetts-lab-tech-arrested...
My agency used to rate our agents based on the amount of cases that led to convictions. Years back, we recognized that disproving an allegation was equally worthwhile, and settings promotions/evaluations based on "proving or disproving" the allegation was much more objectively reasonable than focusing strictly on convictions. There is still a reference to case completion speed, value of recovered money/property, and conviction results (providing a thorough enough investigation that the suspect is convicted for X years, for example), but disproving an allegation or proving that a different person committed the crime is far more appreciable to the agency.
We're fortunate in that we are a federal agency, instead of local/state level, where they can be significantly more busy with countless lesser offenses, while still running some extremely high-profile stuff, but I think it would be a huge boon to law enforcement if states mandated an approach focused on this objective metric that isn't strictly on "getting him".
Also related: we know that funding bias [0] is a real problem in science, despite that scientists' reputations should (ideally) motivate them to resist such biases.
Not arguing just trying to understand.
My rationale is: prosecutors have to weigh an “easy” conviction vs the possibility that labs that always guarantee results may be biased. In the event that they get caught up with a biased lab, wouldn’t it look bad for their reputation? As in, they can’t even perform due diligence on their sources, how can you trust them in a promoted role?
Or am I just naive on how the legal system works?
I am not a lawyer but as I understand the U.S. criminal justice system (which is adversarial between defense and prosecution with the judge officiating and the jury picking the winner) prosecutors are not experts in particular technical fields which is why courts allow expert witnesses. The defense is responsible for calling their own experts who can dispute other biased witnesses. If no one successfully disputes an expert witness over time the prosecutor's trust in them is very likely to grow.
A prosecutor will always claim they did due diligence by selecting a reputable expert witness up until it's proven that the witness was not in fact reliable, but claim (likely in good faith) that they had no way of knowing the expert witness was biased.
A better solution might be to mandate that expert witnesses get independently tested but again that's something a defense council could facilitate and bring into evidence. If no defense team finds it valuable to do this kind of work then it's unlikely a prosecutor will see it as useful either. Lack of challenges implies that the evidence is more or less indisputable from a legal perspective.
How can the truth-finding benefits of the adversarial system be preserved while leveling the playing field by equalizing resources?
My naive thought is that both parties pay into a common fund that is split equally between the prosecution and defense. But that seems like it has it's own set of pitfalls. Are there other models we can look to for ideas?
One idea is that whoever introduces an expert witness must also pay for the other side's expert witness, up to the cost of your expert. The defense doesn't need to prove anything due to the presumption of innocence, so the defense gets free counter-experts and only have to pay for the counter-expert if they want to bring up an expert about something the prosecution doesn't want to talk about.
The real issue is why are there so many cases?
That process happens very rarely. Federal criminal cases are resolved via plea bargain in ~97% of cases and state criminal cases are resolved via plea bargain in ~94% of cases[0].
This is a travesty of justice, especially since most defendants are severely over-charged and often face long prison sentences if they actually insist on (and can afford) a trial.
Those practices, along with cash bail, force even innocent people to plead guilty to avoid having their lives destroyed by bankruptcy, loss of employment, homes and even custody of their children.
And once they have a criminal conviction, they are stigmatized for life and are shut out of many jobs.
In most of these cases, the forensic evidence (if any) is never presented. For a discussion of this, as well as American forensic practices, see Blood, Powder and Residue[1], by Beth Bechky (Author discussion can be found here[2]).
While there are no required standards for forensics labs in the US (with some exceptions[4]), there are accreditation programs (example[3]), and not all forensic laboratories are for-profit companies.
I'm not defending the quality or independence of any particular forensic lab, but it's definitely more diverse than just a bunch of corrupt, rapacious scum sucking at the teat of prosecutors' budgets.
That said, most criminal defendants are at a significant disadvantage when it comes to performing/challening forensic research, as many state and federal labs provide such services for prosecutors, while defendants need to pay, often through the nose for them.
It's just another way the US "justice" system is stacked against criminal defendants.
[0] https://www.nbcnews.com/think/opinion/prisons-are-packed-bec...
[1] https://press.princeton.edu/books/hardcover/9780691183589/bl...
[2] https://www.c-span.org/video/?508130-1/blood-powder-residue
[3] https://anab.ansi.org/en/forensic-accreditation
[4] https://www.justice.gov/opa/pr/justice-department-announces-...
Edit: Added notation for reference [1]
There are plenty of profit motives involved that have little to do with securing future contracts with police departments.
At the most basic - Source code reviews are expensive. They can often throw thousands of false flags that developers have to individually track down and verify are not problems.
Then there's the reputation hit of admitting a mistake - If you've had a bug for the last 5 years that makes you less reliable, that alone is likely to impact future contracts. So given the option of revealing this, admitting the mistake, and tackling the cases it impacted... OR... simply sweeping it under the rug and fixing it internally - some companies will take option number 2 (possibly most companies).
A fairly simple step is to certify equipment independently, and to do blind testing (one sample is provided from evidence; another by an independent unrelated source). Of course this costs more, but anything else is effectively an experiment without a control.
I would argue that if prosecutors and/or cops get to pick the tool, false positives would result in more sales.
Just as false negatives would result in more sales if the defendant gets to make the pick.
That seems highly unlikely. Prosecutors, like all attorneys, talk. It wouldn’t take too many getting burned by a false positive before word would get out (regardless of any nda).
For example, once an expert gets Dauberted, you’ve got to think long and hard about being the next guy to use them.
Sure, in extreme cases it might be obvious, but if you just know that one test gives more matches than another, not knowing which one is the one giving false results?
Every incentive you have leads you to preferring convictions.
You have a choice between two labs to hire for your field test. Lab A and Lab B.
Lab A gives you conclusive evidence leading to a conviction 95% of the time. Lab B gives you conclusive evidence leading to a conviction 65% of the time. Price and speed are roughly comparable between the labs. Which lab do you select?
---
You are now a lab director. You are under pressure to improve the sales of the lab's forensic services. You have pretty much maxed out making changes to improve price and speed, and have been matched by other labs. You understand very well the decision making process prosecutors will use to hire your lab.
---
The incentives clearly lead prosecutors to pursuing labs that deliver better results. Labs know this, and so are under market pressure to delivery convictions for prosecutors.
The ability for an expert witness to deliver more convictions, even if by lying, is a feature for prosecutors, not a bug.
Policing is driven by numbers. There was a whole season of The Wire about this.
Their goals are always to get plea bargain because that is automatic guilt or if they take it to court, they'd better win. That's a lot of state resources just to lose.
Overtime, you'll get people willing to bend the rules, turn a blind eye to 'weak evidence' or whatever.
You're only as good as your measurement/incentivisation.
Even innocent people with a good alibi are going to take the deal.
And if it comes to light, acknowledge a one-time error and carry on.
On the other side a metric which says how many times no conclusive evidence was found with DNA testing sounds more like a metric of failure (but is not, if you don't have qualitative good enough evidence including "complete enough" DNA you should not make up results which simply don't exists, but then courts ruling without conclusive evidence isn't that rare and some would say even necessary to some degree. It's just that many believe DNA tests are somehow unfailable perfects ways to prove the presence of an specific person at a specific place, but they are often not).
It's not that the venn diagram of tech-literate and criminal-defense lawyers are entirely separate circles, but having been "that guy" in a public defender's office for even basic stuff like cell tower triangulation accuracy to finding proprietary surveillance video codecs to decode exculpatory evidence, you really need to start at square one while the prosecution have the whole police department's resources, expertise, and initiative at their disposal. You can afford the expert, it just won't do any good when you don't know what questions to ask that will actually be effective. And unsurprisingly those who do have experience in technical, specialized fields tend to get poached into the private sector or out of trial (really plea) practice all together, so the knowledge/bullshit gap will still exist and there's no real consistent way to bridge it.
Go figure.
Here's a look at two different Texas crime labs. Compare and contrast, and see if you can spot the differences.
https://gritsforbreakfast.blogspot.com/2021/02/beyond-aid-to...
It starts with one pledging to:
aid law enforcement in the detection, suppression, and prosecution of crime
provide medical examiner and crime laboratory services of the highest quality in an unbiased manner with uncompromised integrity.
Their reputation doesn't matter as much as you think. Police procurement isn't based on the reputation of the vendor. It's based on whether or not the vendor delivers the results they want.
In fact, government procurement in general, isn't based on the reputation of the vendor. This is by design.
Society also no longer has tolerance for processes that are not fully auditable and transparent, especially when it comes to our already marred justice system.
Engineers have some notion that it can all be replaced with science. In so far as science can certify the reliability of tests, that is good. But in the end one has to trust the humans between those tests and the courtroom.
A lot of forensic science in itself is essentially phrenology (tooth prints, hair analysis - the non DNA kind, even fingerprinting is of exaggerated value). A lot of labs systematically turn out biased results with generally no expectation of risk, either personally to researchers and definitely not to higher ups.
Ultimately, forensic evidence should be seen as a signal, but not nearly as trustworthy as witness testimony (which in itself is not very trustworthy), despite what many believe.
And this is important for the general public to understand, the people who will participate in juries. The mystique that forensic experts have can make juries give extremely wrong decisions (even bad aquitalls on lack of forensic evidence).
And to emphasize again, I'm talking about the state of the art without the biases being discussed. The biases only compound this problem significantly.
I imagine they're also incentivised to favour a result indicating guilt.