I'm still not convinced these folks aren't just outliers. People in all walks of life give far too much credence to lucky outliers - CEOs are hired that way, ball players, stock portfolios, sports predictors. If you make enough predictions and cherry-pick the best, you can always find a 'miracle man'.
Yes, that was my point. The article failed to make it clear. It is worded such that it could go either way.
I agree they may be outliers. However, I can't dismiss out-of-hand the possibility that they are actually better. Outliers are common... but so are systematic/correlated weaknesses within large organizations.
Indeed, that's such a big problem that there's a sort-of famous book written by a CIA analyst, "Psychology of Intelligence Analysis", that gives a pretty clear breakdown of how and why analysts fail to reason correctly about situations.
It also included examples of how Robert Gates (yes, the later SECDEF) managed to improve analyst reporting by actually reading the reports submitted and figuratively shredding the ones which were bad.
Before that there had been effectively no downside to submitting reports which were crap, intellectually speaking, and eventually things had become just professional-sounding educated guesses.
Large agencies don't have the benefit of small groups self-critiquing their own performance, so they have to try very hard to set and enforce standards of quality. When they don't do this it should not be surprising when quality drops.
This makes me wonder how well analysts do, when they aren't given classified intelligence information. It's entirely possible that the analysts' skills are being hamstrung by information they think they know, but isn't actually true.
An excellent book on just this subject
https://www.cia.gov/library/center-for-the-study-of-intellig...
According to the article, "In fact, she's so good she's been put on a special team with other super forecasters whose predictions are reportedly 30 percent better than intelligence officers with access to actual classified information."
If they mean that after the team was formed, the predictions are still 30% better, than random chance ceases to be an acceptable explanation. If they mean that the team that they formed was 30% better on the predictions used to grant entrance into the team, then your objection still holds. Unfortunately, I don't think the article makes it clear which is the case.
That said, there are definitely some mechanisms that could lead an outside group to higher accuracy on these sorts of things. The two biggest are: 1. The insiders are simply that bad, for whatever reason, and random chance can easily beat them, and 2. The failures of the insiders are correlated and overwhelm their theoretically superior information, and I'd especially finger potential ideological biases there, which could affect both information filtering and the analysis. (It is one thing to believe the world should be a certain way as a result of one's ideology... it is quite another to believe the world must already be a certain way as a result of one's ideology, even when one is staring evidence in the face that it is not. One is noble and laudable... the other somewhat less so.)