Re:A group of 3,000 citizens is making better forecasts than...
by lostcolony on Wednesday April 02, 2014 @10:46AM
(#7515872)
As to the researchers probably know what they're doing, see my original comment and the very first thing I said, "I need to know more". It's not charming cute naiveté, I simply really think that the article tells me nothing about the methodology.
See response here as to why they're still yes/no questions (TLDR; the outcomes are yes/no) - https://news.ycombinator.com/item?id=7515824
Re:A group of 3,000 citizens is making better forecasts than...
by nzp on Wednesday April 02, 2014 @08:32PM
(#7520049)
Well, I didn't mean your comment was a case of naivete, because, TBH, the analogy with my layman physics example isn't perfect, but it reminded me, it's a similar thing: people who are or should be aware that they have only superficial info on something, act as though what they know is all there is to it. The point is, we can't possibly come up with a meaningful critique of this research based on the given article, and, I think, the most reasonable course of action in this particular case is to assume the researchers know what they're doing (considering who's involved).
Re:A group of 3,000 citizens is making better forecasts than...
by AJ007 on Wednesday April 02, 2014 @04:16PM
(#7518649)
I think the only thing to conclude from the article is that normal people can beat CIA analysts. This is similar to the story about monkey stock picking. (near useless FT story url http://www.ft.com/intl/cms/s/0/abd15744-9793-11e2-b7ef-00144...)
Re:A group of 3,000 citizens is making better forecasts than...
by nzp on Wednesday April 02, 2014 @08:44PM
(#7520095)
Hm, I wouldn't say normal people, but subpopulations of normal people (i.e. not individuals but groups). It's an important distinction in this case. But it depends on how CIA analysts' estimates are aggregated if at all. In fact it would be extremely interesting if CIA normally got predictions by averaging multiple analysts' estimates. That would mean normal people with open source intel are beating trained analysts with confidential intel! In that case, we could say normal people beat CIA analysts. But to me this article implies analysts' estimates are not taken as averages. Any CIA analysts here to set the record straight? :)
So, do you really think the researchers in this case hadn't thought of this? I mean yeah, it's OK to doubt and inquire, but come on, this is so rudimentary...
> If the questions are all yes/no questions
The article says (or at least implies) that the subjects give probabilities of "yes" to each question.
EDIT: Oh, and to add, the article clearly says they're not just making shit up and randomly coming up with probabilities. They collect open source info on events and base their estimates on that. So it's not just blind guessing.