News for hackers. Stuff that matters.
Re:Court rules drug dog barely more accurate than a coin fli... by gburt on Friday August 07, 2015 @12:53PM (#10023502)
This isn't being fair. I don't really think the post title is fair either. That 93% isn't comparable to the 50% you're probably thinking for the coin flip, or even the 0/100% mentioned in your edit.

---

93% alert rate means:

93% = false positive + true positive

7% = true negative + false negative

---

We're given "His success rate was much lower, at 59 percent." Assuming that means that 59% are true positives, we know the false positive rate is 34%.

Without data on how many passengers are carrying drugs, we can't say anything about the rest of that result. A plausible world where the drug dog may be valuable is when he unquestionably rules out that 7% of the population that he doesn't alert on, as in false negative = 0.

I believe the reason this number is so high (59%) is that the officer has already used his suspicion to determine filter out most people. This makes the value of the "ruling out" plausibility even more valuable: the drug dog reduces searches compared to trusting the cop.

Ideally, they'd provide the receiver operating characteristics curve [0] or the confusion matrix [1].

[0] https://en.wikipedia.org/wiki/Receiver_operating_characteris... [1] https://en.wikipedia.org/wiki/Confusion_matrix

Re:Court rules drug dog barely more accurate than a coin fli... by nickff on Friday August 07, 2015 @01:00PM (#10023541)
The real question is whether the dog is better than the officer at guessing who has drugs. If the dog is much better than the officer, it would suggest that a dog 'alert' should be given more weight than the officer's guess. If the dog is no better than the officer, the dog is useless.

I say this because it is possible (and indeed likely) that the officer is also better at guessing who has drugs than a coin toss.

Full Disclosure: I have never used or trafficked contraband, but I do not support random searches or the use of drug dogs in public without a warrant, regardless of their effectiveness.

Re:Court rules drug dog barely more accurate than a coin fli... by LordKano on Friday August 07, 2015 @01:09PM (#10023578)
If the dog is much better than the officer, it would suggest that a dog 'alert' should be given more weight than the officer's guess. If the dog is no better than the officer, the dog is useless.

A very big problem is that drug dogs will alert when the handler gives it subtle clues. This is done to get probable cause for a search when there is nothing more than an officer's hunch or just a desire to hassle a particular individual.

Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @01:16PM (#10023620)
This is something I see claimed a lot. I'd be very interested in any information you have about this. I find it very unlikely this kind of training is part of the official training, so I'm imagining it's some kind of unintentional conditioning? Or is it done on purpose by the officers?
Re:Court rules drug dog barely more accurate than a coin fli... by lukeschlather on Friday August 07, 2015 @01:31PM (#10023714)
This is covered in the article, and it's quite straightforward. The handler was giving the dog treats for alerting, rather than giving treats for finding drugs.

The primary trouble I see is that there is no penalty for re-training dogs like this.

Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @02:35PM (#10024136)
The article only discusses that the handler gives the dog treats for alerting, it doesn't go into the motivations about that, whether they're because the officers aren't considering the Pavlovian conditioning involved and are accidentally reinforcing such behavior or if it's intentional.
Re:Court rules drug dog barely more accurate than a coin fli... by kd0amg on Friday August 07, 2015 @02:50PM (#10024245)
Does the motivation matter when we're talking about whether it establishes sufficient justification for searching someone's car? After a clear flaw is pointed out, a response of "We didn't mean to ruin the statistical power of this test" wouldn't do much to restore my confidence in its results.
Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @03:13PM (#10024385)
I was specifically responding to this comment

>This is done to get probable cause for a search when there is nothing more than an officer's hunch or just a desire to hassle a particular individual.

that claims that this is done purposefully. That's what I was wondering about, not the overall efficacy of drug dogs.

Re:Court rules drug dog barely more accurate than a coin fli... by radiorental on Friday August 07, 2015 @01:35PM (#10023738)
You can watch this 6 minute video of a stop, I've linked to where the officer was caught triggering the search

https://youtu.be/w-WMn_zHCVo?t=3m35s

It happens.

Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @02:31PM (#10024117)
Thank you. It is kind of the videographer's assertion that "Check here" is a command to trigger instead of just a command to check there, however, isn't it?
Re:Court rules drug dog barely more accurate than a coin fli... by radiorental on Saturday August 08, 2015 @07:38PM (#10028977)
Yes but no drugs were found and the 'scratching at window' was the trigger for search. As the op article indicated - four legged search warrants.
Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Sunday August 09, 2015 @05:11AM (#10029922)
That doesn't really address my comment. It's still just an assertion that that is an intentional trigger. The op article similarly makes this assertion but there is no supporting evidence that I saw.
Re:Court rules drug dog barely more accurate than a coin fli... by jlarocco on Friday August 07, 2015 @01:40PM (#10023761)
It's not intentional, but it's a real thing:

http://www.ucdmc.ucdavis.edu/welcome/features/2010-2011/02/2...

Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @02:38PM (#10024161)
So it's less an issue of police officers intentionally triggering things in order to harass individuals like the original commenter said and more of unintentional training. Thank you for the link!
Re:Court rules drug dog barely more accurate than a coin fli... by LordKano on Friday August 07, 2015 @03:44PM (#10024570)
One can call it plausible deniability.

There are many examples that have been caught on camera, I see that another commenter posted a link to one of them on this thread.

When you see things like the handler returning to a previously cleared spot and indicating that the dog should check it again; three or four times until the dog alerts, you can assume that the handler has a severe short term memory problem and he just forgot that the dog had previously checked the area and failed to alert. I think it's more indicative of the officer hinting to the dog that "this is where I want you to alert" and their training reinforces the idea that if the dog does what's expected, there will be a treat.

Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @03:57PM (#10024630)
That's why I was asking about if there was any information about this, or any studies/investigations done, since people can go back and forth forever with "Well, did they already check that part? Is it intentional? Was the dog triggered?" Etc.. etc... It definitely seems like an area ripe for investigation, I just haven't seen any.
Re:Court rules drug dog barely more accurate than a coin fli... by marincounty on Friday August 07, 2015 @01:58PM (#10023907)
Antidotial evidence: I have never had a dog that didn't know my moods, fears, apprehensions, etc.? I don't think being called "Man's best friend" is a worn out cliche?

My last dog picked up on so many of my personality traits/tells-- I felt quilty.

(My cat--whole nother story.)

Re:Court rules drug dog barely more accurate than a coin fli... by mejari on Friday August 07, 2015 @02:36PM (#10024146)
Very true, I guess my question is about the intentionality of it. I doubt you intentionally trained your dog to respond to your moods, but he/she did it anyway. The claim here is that officers are doing this intentionally in order to get around the law of probable cause. I'm interested in cases where this has been shown to happen.
Re:Court rules drug dog barely more accurate than a coin fli... by gburt on Friday August 07, 2015 @01:02PM (#10023551)
I have no opinion on the legal and ethical issues, I was commenting only on the classifier. I don't think your first question is appropriately described as "the real question" though, as one of the coolest results of machine learning is that there are techniques for combining multiple bad classifiers [0] in to one really good one.

Even more so, I think that is actually what is happening here. An officer is acting as a tier 1 filter, and then the drug dog is acting as a tier 2 filter, further reducing the number of false positives, which in a "legal and ethical" sense, we generally feel are worse than false negatives.

[0] https://en.wikipedia.org/wiki/Ensemble_learning

Re:Court rules drug dog barely more accurate than a coin fli... by nickff on Friday August 07, 2015 @01:14PM (#10023609)
>" An officer is acting as a tier 1 filter, and then the drug dog is acting as a tier 2 filter, further reducing the number of false positives, which in a "legal and ethical" sense, we generally feel are worse than false negatives."

If the officer is acting as a filter, then it seems to me like this is a search, and not allowable, because it would be subject to all the usual objections to random searches. You might be right, but if that is the only way for these searches to be better than the officer's guesses, it seems that drug dogs would not be allowed to 'sniff' in public without warrants.

Re:Court rules drug dog barely more accurate than a coin fli... by jeremysmyth on Friday August 07, 2015 @01:09PM (#10023580)
It's in the article. The 93% in that study includes a 59% true positive (of 93%, presumably) success rate. We can't know how many true or false negatives there are because those searches aren't actually carried out, so out of every hundred cars stopped for a dogsniff, 55% (59% of 93%) are found to be carrying drugs.

More worryingly, that also means that out of every hundred cars stopped, 38 are wastefully searched by police (after getting probable cause from the positive dog alert) without finding drugs. From the article, a search takes about 45 minutes and carries a bunch of other risks for occupants of suspected cars (e.g. civil forfeiture, overenthusiastically trigger-happy LEOs), so this is pretty bad situation, not just in wasted time and cost, for both the car occupants and the police force.

One slightly positive thing I'll take from this: Over 50% of cars that are dogged have drugs, and I sincerely doubt that over 50% of cars (on average) have drugs, so in this particular study it appears that police have a better than random chance of guessing which cars have drugs. It's not a very positive thing though, because civilized countries that operate under the rule of law cannot permit law enforcement to operate on guesswork. This is underlined by the fact that the article mentions that the false positive rate rose to something like 80% for latino drivers.

Re:Court rules drug dog barely more accurate than a coin fli... by gburt on Friday August 07, 2015 @01:13PM (#10023605)
Sorry - I edited my post multiple times after posting it to clean it up and increase quality. Some of your response (especially the first paragraph) is now confusing. That was my fault.

I agree with you mostly - especially on the discussion of what "probable cause" should be defined as and what constitutes a "random" search, as just an example of some of the ethical issues of interest here. That is a debate to have and even one I think this community is well equipped to discuss, but it was not something I was trying to comment on.

I was only making the point that it wasn't fair to jump on the 93% as though it was representative of the problem. I am not taking any ethical or defensive position of drug dogs here.

Re:Court rules drug dog barely more accurate than a coin fli... by brobdingnagian on Friday August 07, 2015 @01:07PM (#10023573)
The dog has low precision and high recall. The dogs are capable of high precision and high recall, but aren't being trained to do that because other incentives are at play.

I found it impressive that the dog that alerts 93% of the time was correct 40% of the time. That means the police are doing what seems like a good job of identifying targets that actually have drugs. With those kind of odds, you'd probably want to take a closer look at everyone just to make sure you aren't letting people with drugs get away (false negatives).

Re:Court rules drug dog barely more accurate than a coin fli... by mikeash on Friday August 07, 2015 @01:12PM (#10023595)
It's plausible that the 7% is all true negatives. It's also plausible that the 7% is carrying drugs at the same rate as the 93%, and the drug dog is doing nothing at all. If that is the case, then flipping a biased coin would be just as effective as the drug dog.
Re:Court rules drug dog barely more accurate than a coin fli... by okasaki on Friday August 07, 2015 @01:08PM (#10023574)
Erm, let's say 10% of the people on whom drug dogs are called have drugs.

Then for 1000 stopped, you'll have

0.59 * 0.1 * 1000 = 59 true positives and ...

0.34 * 0.9 * 1000 = 306 false positives.

That's pretty terrible.

Re:Court rules drug dog barely more accurate than a coin fli... by harryh on Friday August 07, 2015 @01:16PM (#10023619)
Your math is wrong.

The article indicates that of all the people the dog indicated had drugs 59% of them actually did. So for 1000 people whom drug dogs are called on

930 the dog indicated had drugs. Of those 549 were correct and 381 were false positives.