The fact that that polygraph tests don't work was well known certainly by the 90's (if not much much earlier). I struggle to believe anyone at all interested in the field wouldn't have some awareness of that, especially if they knew in advance they were going to be tested.

And I'd dispute that they "work" even if the subject is unaware of the flaws. Studies have shown innocent people regularly react to assertions of criminality. It's basically left up to the individual tester to "interpret" whether or not they find the person to be lying.

Sure sometimes it may inspire someone to admit to something they otherwise wouldn't. But these people are more likely to be largely honest people who made a mistake then hardened criminals or people actively working against the state.

>Studies have shown innocent people regularly react to assertions of criminality.

Yeah, if I were accused of a big crime I would look at it like this: if my faith in the justice system is 99%, and if I'm facing 50 years in jail, then given my innocence I'm facing an expected value of 6 months in jail. You can bet I'd be sweating!

How is this different from a clinician who has to diagnose autism without even such a handy instrument.

Colloquial view: Presumably someone going to see a psychiatrist about autism in interested in an accurate outcome as well as the doctor.

The main test they use for ASD, the Autism Diagnostic Observation Schedule (ADOS) – a Cochrane review [1] found that across 12 studies of diagnosis in preschoolers, the ADOS-1 [2] had a summary specificity of 0.8 – in other words, an average false positive rate of 20%. But, looking at the studies individually, the specificity ranged from 0.2 to 1.0 – in other words, the false positive rate ranged from 0% to 80% across the individual studies. And, that is in research contexts – there are good reasons to believe that the test will be less accurate in clinical use than in research contexts. In order to use the ADOS in research contexts, one has to demonstrate "research reliability", which is a process of scoring videos of the test being performed and comparing your scores with one's colleagues, repeating the process until you get sufficient agreement, and doing this again once every few months to make sure you stay in agreement. None of that rigour is required in clinical use. Added to that, I think in clinical contexts psychologists sometimes feel the urge to "put their thumb on the scales", they'll think "I know the test says this child doesn't have ASD, but I know they are struggling and an ASD diagnosis will get them funding that will help them, so I'll just fudge the numbers a bit to make it come out positive." Some psychologists who would never do that in research (it's scientific fraud) will do that sort of thing clinically. So, it seems very likely (if difficult to prove rigorously) that tests like the ADOS will have less accuracy in clinical contexts than research contexts, so however accurate they are measured to be in these studies, it is probably worse than that in real life.

Sometimes, they suggest combining the ADOS with the ADI-R (Autism Diagnostic Interview-Revised), which is a structured interview of parents/caregivers. The combination is even called the "gold standard", with the suggestion that using the two in combination is more accurate than either alone. But, does the research actually support that belief? Well, that Cochrane review says they could only find one study (in preschoolers, they excluded studies of older individuals) which compared the accuracy of ADOS-1+ADI-R to ADOS-1 alone. It found a modest improvement in specificity (equivalent to false positive rate), at the cost of a modest worsening in sensitivity (equivalent to false negative rate), but the result wasn't statistically significant. So, this belief that using the combination improves accuracy doesn't appear to really be supported by the evidence.

But, do you think clinicians tell parents any of this? In my personal experience, no. Either they are ignorant of the research, or they are keeping that knowledge close to their chests. How much faith would parents put in the results of the ADOS if they were told that in research contexts it had a 20% false positive rate on average? And that there is good reason to believe that in clinical contexts (their current context) the test probably performs worse than that? And that 20% is just an average over multiple studies, and in some individual studies its false positive rate was substantially higher?

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513463/

[2] the ADOS-1 has been replaced by a new version, the ADOS-2. At the time of that Cochrane review, there hadn't yet been any studies published on using the ADOS-2 in preschoolers, so it was excluded from their analysis. But, the fundamentals of the test haven't changed, so I doubt the ADOS-2 performs significantly better than the ADOS-1 did.

I agree with the first part but the second doesn't really follow. How will your belief that phrenology doesn't work influence the result of such a test (assuming you live in a world where such tests are even considered)?

Phrenology doesn't allow the same "luxuries" as the polygraph because people are more or less stuck with the size and shape of their skull. Their guilt is exclusively in a chart not in their beliefs. Only thing that matters is if the people (judicial system) giving the test believe in it.

Ok but are applicants for intelligence jobs that naive/uninformed? I’m not particularly intelligent or informed but I’ve come across articles and books debunking polygraphy many times over the past 20 years. It’s not very obscure information. Scary to think that our “best and brightest” can’t be bothered to do cursory research. Also if I was facing a polygraph you can be damn sure I’d research it ahead of time. Are candidates not doing this?

Even being aware that something is fake doesn’t necessarily mean that you believe it.

There are placebo effectiveness studies where the participants know they are taking a placebo pill, for instance, and yet still feel better than a control group who isn’t offered any treatment. https://www.health.harvard.edu/blog/placebo-can-work-even-kn...

This seems like the right answer. If the polygraph is able to essentially determine some sort of nervousness beyond normal, then someone who really believes they work and is not used to lying is going to be really damn nervous trying to lie and get caught up.

But if someone knows what it really does, and then just knows you just have to be calm and say whatever you want, then it won't work.

These are supposed to be smart people. Why would they believe it works? Wouldn’t they google polygraph tests before showing up?

What makes you so sure there isn't some sort of classified biomarker or correlate?

If it’s some other “secret” test then we’re no longer discussing the polygraph are we?