Consider a drug sniffing dog at an airport, it sniffs all carry-on bags as people go through the line to be searched. If we find drugs even 10% of the time that the dog whines, it's amazing, because the actual distribution of drugs amongst travelers is minuscule, maybe 0.01 of 1%.
But if the dog is right 55% of the time? This is excellent probable cause for a search.
But by way of contrast, consider a drug-sniffing dog that is only brought in when a human handler suspects a traveller may be carrying drugs. If the handler is very good at their job, and optimizing for success, the handler may themselves have a 55% positive-positive rate. Meaning, 55% of the people they pick are carrying drugs.
Now if the dog whines 55% of the time they are given a suspect that has a 55% chance of carrying drugs, well, that is no better than a coin flip, it's garbage.
And of course, what if the handler is amazing at picking people who might be carrying drugs? For example, what if they have access to warrentless surveillance meta-data and are using the dog as an excuse for parallel reconstruction ("Your honour, I picked this person at random, and the dog whined, so I searched them"). The handler might have a near 100% success rate of picking people carrying drugs.
Now a 55% rate of whining is truly terrible for deciding whether the dog is any good.
Going back to the original case, even if we assume that every time the dog doesn't signal there wasn't drugs (so the dog never missed a drug), had the police just searched without bringing in the dog they would've had similar rates. So why isn't the police's 'I think he had drugs, can't give any good reason though' not probable cause if the success rate is nearly 55% when the expected success rate of searching everyone is under 1%?
There is some aggregate base rate, which is the percent of a random sample of motorists that are carrying drugs.
Then there is the rate of those who are actually carrying drugs within the a group that the police have pulled over, and suspect are carrying drugs, and have also presumably asserted their rights and refused a consensual search by the police and also do not have some other attribute (such as smell, items visible in plain view) that would provide probable cause.
You argue that "the base rate for drug possession in a car is almost certainly a tiny fraction of that" but in fact the base rate within this much narrower second group should be extremely high in comparison to the general population. And the problem is that searching that group, regardless of what you think of them, is illegal.
In order to make it legal you have to be able to show that you're going to do something specific that will provide probable cause.
Showing that the dog is anything short of a clearly reliable and accurate means of determining that a law is being violated means that the dog is not providing probable cause. A dog that alerts to drugs more 90% of the time when only 60% or so of the population in question has the drugs, has shown results that aren't sufficiently correlated to the data it is supposed to discern.
When that is true it is just a tool for violating laws that prohibit searches.
Yes, but that hurts your argument rather than helping it. The drug dog does not, by itself have to clear the probable cause hurdle. Rather, the totality of the circumstances has to yield probable cause: https://en.wikipedia.org/wiki/Illinois_v._Gates.
So say the base rate for carrying drugs is 10% in the population. The police, based on their suspicions, pull over a population of people where 40% are carrying illegal drugs. Then, the drug dog comes in, and he's barely better than a coin flip--60% correct. Out of 100 people stopped, he'll correctly identify 24 of the 40 carrying drugs, and incorrectly flag 24 of the 60 not carrying drugs. But overall the process will be right 50% of the time, versus only a 10% probability if you stopped drivers at random. That's probable cause.
For example, let's say that the dog is not sufficient probable cause. Consider the cases:
1) drug dog indicates + suspect driving erratically
2) drug dog indicates + "I just have a feeling"
Am I correct that 1) might be PC but 2) could not be, no matter how accurate that officer turns out to be in practice?
(In case 2, the dog's finding more drugs than the base rate would not validate its accuracy, as the results are being pre-filtered to have a very different distribution.)
So rationally, the cop's "spidey sense" is indeed evidence. The question I posed to you (and whoever's reading) was whether it counts as legal evidence. And it's my understanding that, no matter how good your spidey sense is, it does not become PC in a legal sense until it is articulable. As Thomas Moore said in A Man for All Seasons:
"The world must construe according to its wits; this court, according to the law."
https://en.wikipedia.org/wiki/Receiver_operating_characteris...
Now, what's the actual base rate? What proportion of cars on the road contain drugs? I'd wager it's well under 10%. Even if we say it's 10%, that means that the humans are taking the rate from 10% to 55%, and then the dog goes from 55% to 59%.
In other words, the dog is basically useless, except as a tool to make the searches legal. If not for the legal requirement, one could just as easily say that "between the human judgment to bring in the magic 8 ball, and the magic 8 ball itself, the police are able to find drugs at probabilities that are multiples of the base rate."
Apparently police are really good at figuring out whether cars contain drugs, but the law doesn't allow them to act on this directly. And apparently dogs are nearly useless at figuring out whether cars contain drugs, but the law allows their nearly useless indication to be used as the basis for action. This is rather backwards!
One wonders if we have the procedures we do because it gives a thin veneer of scientific respectability over a system that might otherwise appear biased, racist & otherwise distasteful even if it's statistically valuable.
There does seem to be a lot of "this is 'scientific' so it's good" in law enforcement. A lot of forensic techniques are just BS, but are accepted because they're "scientific." Or look at the recent article posted to HN about how polygraphs are nonsense.
Of course, it's the job of the police and the courts to come up with an answer, so we should be able to let them do it. But it seems they can't be trusted to reason about it well.
If all you need is your own suspicion, it could be easy to let one's bias against purple people run rampant, and search every purple person's car and nobody else's.
If you have to call in a dog which you believe to be effective (even if it may not be) then you may only go to that trouble when you actually have some legitimate suspicion, rather than just going based off your anti-purple racism.
No, it isn't. 'A common definition is "a reasonable amount of suspicion, supported by circumstances sufficiently strong to justify a prudent and cautious person's belief that certain facts are probably true"' https://en.wikipedia.org/wiki/Probable_cause#cite_note-2
I would argue that if you roll a 100 sided dice, my belief that the result is over 45 is not 'prudent and cautious' but 'optimistic'.
The point of that analogy was to create a situation in which someone is right 55% of the time and see how the language "prudent and cautious" feels in those circumstances.
Whether that means rolling over a certian number on a die, or rolling a specific number on a very strange weighted die is irrelevant.
Consider, instead, something like guessing whether someone has had cancer. Only 8.5% of the population has ever had it. If you can guess randomly, you'll be right about 8.5% of the time. If you can guess correctly 55% of the time, you're doing far better than random guessing.
The implicit assumption of the "probable cause" concept is that the base rate is low. I.e. only a small percentage of people have illegal contraband. If a police officer is able to guess that someone is more likely than not to have contraband (i.e. more than 50% correct), then he must be acting on some concrete clues, because he's doing far better than if he just searched people randomly.
"The problem here is that invasive searches based on no more than a government official’s hunch is precisely what the Fourth Amendment is supposed to guard against."
Due to the high alert rate, we actually have a pretty good idea of the base rate of drugs in the situations that this dog is being used in (~50-60%).
The question is whether the dog is actually providing probable cause when the dog has such a high false positive rate.
The assertion of the article is specifically that drug dogs are being used and trained by police in ways that allow them to be be used as "a probable cause generator".
"The Supreme Court originally gave its imprimatur to drug dog sniffs because when used properly, the dog’s finely tuned sense of smell can detect drugs and their absence with incredible precision. But that caveat — when used properly — is critical. If a drug dog isn’t eliminating any innocent people, if it’s validating the suspicions of the police nine out of ten times resulting in searches for which up to half or more of the suspects are innocent, then descriptors such as finely tuned and incredible precision no longer apply."
Thus the base rate that we should be judging the dog against is not the base rate in the general population, but the base rate in the dog's tested sample. We need to incentivize the cultivation of higher levels of accuracy that the dogs are capable of providing.
Now if you take the 50% of people you guessed to be positive for test, only 4.25% will be true positives, indicating 8.5% accuracy rate; if you take the 50% people you guessed to be negative for test, 45.75% will be true negatives, indicating a 91.5% accuracy rate.
But your overall accuracy rate is 50%.
edit: Seriously? A downvote for this comment?
I downvoted your comment because the argument you are attempting to make is incoherent. I'm sure that you understand what you are trying to say, but as written your comment does a poor job of expressing that point of view.
If have a procedure for guessing the number rolled on a 100 sided die, you are choosing between 100 different possible guesses. That is nothing like a dog guessing if a person has, or does have drugs (two possible guesses). Attempting to compare the accuracy rates between those two cases is major statistical error.
I thought this difference would be obvious to most readers of HN and was thus surprised to see such a simple statistical error being made. I did not think that verbose explanation would be needed once the error was pointed out.
But I don't think "prudent and cautious" can quite be boiled down to just a certainty of outcome. Risk/reward plays in a lot. If we take "cautious" to mean a medium degree of risk aversion, then I don't think a cautious person would risk their year's wage on the chance to double it, even with a 55% certainty of success. However, I think they would risk $10 for a 55% chance of getting $20.
So what is at stake? What's the cost of an invasion of privacy?
Because of civil asset forfeiture, the base value is however much cash happens to be in the car, assuming it's enough to be "suspicious" (how just having cash is suspicious, I don't know). Most people don't have large amounts of cash in their car, but those who do have, well, a large amount.
The other quantifiable loss to a car search is time. Based on a quick search, it seems the average hourly wage of americans is $25. Wage isn't really the best evaluation of time, but it's easy. That covers the time wasted, but not the losses due to being late to something. A car search could make someone miss an appointment, or even get fired — or nothing.
I'm going to estimate that the sum of how much the average person stands to lose to asset forfeiture and time is about $100. That's of course not even considering the emotional impact of having your privacy violated, or the potential non-legal consequences of having a car search, say having an affair exposed. I'll try to give a personal answer to that question. How much would someone have to pay me for me to be okay with them tipping of the police that I have drugs in my car? Well since I'd be able to plan for it, missing appointments and asset forfeiture are hardly a concern, but time is. I suppose someone could hire me to stand around and be bored for an hour for $40. I'd let them tip off the cops for say, $600. Since that's how much it would take to make it worth my while and not really just meet the baseline of acceptable, I'll cut off a bit from that, say $60. And then subtract the $40 for my time. Adding in the estimated tangible cost, and that's $600 again.
So let's go with $600 as the cost of an erroneous search. Now we need the value of a successful search.
It's hard to quantify (I'd argue the value is negative because of the questionable value of the war on drugs and the cost of incarceration), but for this purpose, I'll use bounties. A quick search turned up a really bullshit looking page that mentions a county police department offering $1000 cash to anyone who turns in a heroin dealer. While the value to the county is going to be higher than the bounty they offer, I'm going to assume that since this county is offering a bounty in the first place, they value it slightly higher than other municipalities. So let's go with $1000 as the value of catching a drug dealer.
Risking $600 for a 55% chance of $1000 isn't just not cautious, it's a losing game.
*
So what would I consider cautious? Risking $1000 for a 60% chance of doubling it? I'd characterize that as "cautiously optimistic". 65%? Same. 70%? Sure I'll call that "cautious". Obviously different people will have different ideas of what "cautious" means.
But by my definition, and my value estimates, I'll say the certainty required for probable cause for searching a car with drugs is 85%.
The main gist of which is the difference between 'concrete' and 'statistical', which seems to be very different from the way many of use think about 'reasonable'.
The article certainly indicates that you don't need 85% certainty to have probable cause. While officers can have pretty high accuracy rates to their guesses, they need a 'concrete' piece of evidence to point to so they can have probably cause. Drug dogs are being used to create this concrete reason, but this has more to do with their perceived reliability than their actual effectiveness reliability.
My biggest concern with the points that this article raises is that it seems like the dogs are being trained to reliably provide probable cause rather than being trained to provide reliable probable cause.
So the article is correct as written. About four out of every 10 roadside searches conducted on the basis of a dog alert are groundless. This outcome is nearly indistinguishable from a coin flip conducted by the officer who has already flagged the car as a suspect vehicle.
> In U.S. v. Bentley, we see just how damaging the Harris decision really was. Lex, the drug dog that searched Bentley’s car, had a 93 percent alert rate. That is, when Lex was called to search a car, he alerted 93 percent of the time. He was basically a probable cause generator. His success rate was much lower, at 59 percent.
In other words, drugs were found in 55% of cars in which he was brought in.[1] The base rate for drug possession in a car is almost certainly a tiny fraction of that. So between the human judgment to bring in Lex, and Lex's sniffing, the police are able to find drugs at probabilities that are multiples of the base rate. In contrast, if you searched cars based on a coin flip (i.e. randomly), you'd expect to find drugs exactly at the base rate.
[1] Being successful > 50% of the time is pretty much the definition of probable cause.