This could easily be way off. It's not testing the people who aren't home at all, and it has a low chance of only testing the people who leave home rarely, only when strictly necessary. It's mainly finding the people who leave home a lot.
If the studied presume this then it makes more sense.
And you completely missed the point that people who are going outside a lot more often are both (a) more likely to have been exposed to COVID-19, and (b) more likely to be encountered by your survey. If there are e.g. 700 daily shoppers and 700 weekly shoppers going to a given store, then if you go on one day and sample every shopper you're not going to get half and half by population as you'd need to for a true random sample; instead, you're going to get 700 daily shoppers and only 100 weekly shoppers! This is a hugely biased sample!
I find it interesting that this study corresponds to an IFR around 0.5%, similar to the IFR arrived at by other studies using very different methods. Perhaps the statisticians who conduct these studies actually know what they're doing better than Hacker News commenters.
And perhaps it's time to accept the politically inconvenient scientific truth these studies are revealing.
With the wild range of IFR from so many studies, you could trivially cherry pick supporting studies for a bunch of different numbers, especially if you throw in am error range.
For my wife and I and kid, we live in SF and order food when we can from grocery. When we can’t, I put on an N99 mask and gloves and buy 2 weeks worth of groceries. Then go through a ritual to be as safe as possible entering the home and bringing groceries in.
If someone asks me at the grocery store to take the covid test, no way. How would I even do that safely???
I live in San Francisco though and am worried about the fact that the grocery is 70% instacart workers.
Everyone I know in the city (nyc, sf, Boston, Chicago etc.) is trying to use 3rd parties to get their groceries.
Everyone I know in the suburbs is going to the store.
I believe the reason people read this study differently is that their experience is different. The whole foods in downtown sf would be scary to be in without a mask. And I view it as the riskiest place I have to go.
It’s full of gig workers who are taking way more risk than the average person.
We will know more when we see the results of the study, but I think we need a more random sample before thinking this NYC study is even remotely close to showing that 20% of NYCers previously had it
This is a sampling bias, and it's the exact same problem as if you ran a survey trying to ascertain how many people have landlines ... by calling random phone numbers.
These bias issues are well known among survey designers. There are ways that you can try and account for them. It seems like maybe the medical community is too used to having control groups in their studies?
No. I'd expect people who order deliveries online and can do their entire job on Zoom/slack all day to be a minority to the point where including them in the study would be covered in a sampling error.
This is not some snooty tech point of view - this kind of flaw in statistical studies matters a lot. The more important the policy implications are, the more important the validity of statistical methods.
It’s anything but parochial.
And even if it were--- it's notable that it's this easy to find a subgroup with cleared infections this far above what was anticipated to be found across NY/NYC.
And-- it's remarkably consistent with statistical estimates made through a variety of other means, e.g. https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v...
This is deeply flawed.
I live in NYC and have been making donations to have food delivered to elderly people since this started.
Just the other day, de Blasio announced that taxi and rideshare drivers would be paid by the city to deliver food - and city agencies would be serving as overflow dispatchers and tech support for it.
Many, many people do not have to go to the grocery store, or they have their healthy younger family member go in their place, etc.
To boot, as Gelman has written on his blog, these studies do not agree with a wide range of other studies, and in most cases the test specificity is on par with the incidence rate itself, making incidence rate estimates from _all_ of the studies very unreliable for correlated reasons, so that pooling the studies does nothing to overcome the huge uncertainty they suffer in incidence rates.
????? We can't exclude that specificity may be 97% (it's unlikely, based on our data, but it's at the edge of the confidence interval), which is unfortunately on par with the studies returning 2-4% positives... but can't exactly explain a return of 21%. Subtract off 4% worst-case false positives from 21%, and where are you?
Since you want to appeal to authority with Gelman, this is what he said about this on his blog: "– Those California studies estimating 2% or 4% infection rate were hard to assess because of the false-positive problem: if a test has a false positive rate of 1% and you observe 1.5% positive tests, your estimate’s gonna be super noisy. But if 20% of the tests you observe are positive, then the false-positive rate is less of a big deal." ... "– In any case, the 20% number seems reasonable. It’s hard for me to imagine it’s a lot higher, and, given the number of deaths we’ve seen already, I guess it can’t be much lower either."
> so that pooling the studies does nothing to overcome the huge uncertainty they suffer in incidence rates.
The case count multiple from the serological study for New York state (and indeed, the California counties) is right in line with what's expected with a variety of statistical measures that were made without relying upon the serological data. So if you even peeked at my source you'd not be making this argument. https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v...
> First off, 3% does not sound implausible. If they said 30%, I’d be skeptical, given how everyone’s been hiding out for awhile, but 3%, sure, maybe so.
But it's not just, say, taking a bunch of faulty studies and average them and saying voila. They are in agreement. They'd have to be bad in similar ways in order to converge like that.
That's unfortunately not the way statistics works. Combining multiple bad tests just makes the results incorrect or highly uncertain; the devil often is in the details. The Stanford/Santa Clara County study is a good example of how the details can really undermine a study.
Things do indeed seem to be converging to the idea that the CFR is not near the IFR, but none of this is new news, and the IFR remains very close to what most epidemiological predictions indicated early last month. If anything, it confirms that COVID is a difficult beast to tame.