News for hackers. Stuff that matters.
Re:The Parable of Google Flu: Traps in Big Data Analysis [pdf] by Fomite on Monday March 24, 2014 @12:49AM (#7456768)
This. A failure to match the CDC's confirmed cases is a failure of the Google model, because that's what it's trying to match.

It also fails fairly spectacularly on a local level when compared to health department data from some top flight state agencies, like NY.

Re:The Parable of Google Flu: Traps in Big Data Analysis [pdf] by colin_mccabe on Monday March 24, 2014 @02:31AM (#7457020)
The CDC's model "fails pretty spectacularly" when it comes to capturing any of the times I've had the flu. I have had the flu several times and the CDC has never been aware of it. As I explained earlier, I never told my doctor that I had the flu or went to a lab. My local health department was never aware of these cases either. It might, however, have appeared on Google flu trends, since since I might have searched for what the impact of a certain level of fever was. It's been a few years so I don't remember if I did or not. And of course, Google has never published which search keywords they use.

There are three datasets here: the CDC's, Google's, and reality. Some folks here seem unable to differentiate between the CDC data and reality. But in fact, I really did have the flu, even though CDC didn't think I did. Reality is what matters.

Maybe there is reason to believe that CDC's data is closer to reality than Google's. If so, let's hear it.

Re:The Parable of Google Flu: Traps in Big Data Analysis [pdf] by Fomite on Monday March 24, 2014 @11:22AM (#7458711)
The key is, like all population studies, the CDC doesn't have to know about you. It has to know about some people like you, and honestly, if you were well enough to not need to go to the doctor, it's also not really the influenza public health authorities are worried about.

The CDC is unapologetic about their data being estimates - but they're very solid estimates, and match more intensive but smaller scale studies pretty well. But we don't need to capture every single flu case - no surveillance system will ever do that, nor need to.

Re:The Parable of Google Flu: Traps in Big Data Analysis [pdf] by colin_mccabe on Monday March 24, 2014 @11:07PM (#7463558)
I just feel like either I am missing something, or there is a lack of rigor here. Clearly, sampling introduces errors and under-reporting introduces errors. What I am looking for is a case (based on data, not hand-waving) that Google's errors are worse than the CDC's, and so far I'm just not seeing it. Maybe I am missing something which is obvious to people in the field.

Edit: I see how the CDC's numbers could be a lower bound, but not upper.