News for hackers. Stuff that matters.
Re:Preliminary test results suggest 21% of NYC residents hav... by WrtCdEvrydy on Thursday April 23, 2020 @02:16PM (#22959081)
Unfortunately, unless you go door to door knocking... you're gonna get biased data.
Re:Preliminary test results suggest 21% of NYC residents hav... by ineedasername on Thursday April 23, 2020 @03:02PM (#22959728)
Or truly random sampling that includes appropriate proportions of sub groups, e.g, socioeconomic status etc. Otherwise you have to know on all of the doors. But it's still a lot: for a 95% confidence level and 2% margin of error, for NYC you would need to test about 2,400 people. Assuming a response rate of 10%, you need to knock on 24,000 doors.
Re:Preliminary test results suggest 21% of NYC residents hav... by austingulati on Thursday April 23, 2020 @03:54PM (#22960417)
Wouldn't the ability to refuse taking the test introduce bias? i.e. the 10% that do take it still may not be representative of the entire population. Likely better than the grocery store sample, but still not ideal
Re:Preliminary test results suggest 21% of NYC residents hav... by qqqwerty on Thursday April 23, 2020 @04:58PM (#22961310)
Yeah, that is still an issue. You can offer compensation to incentivize participation. And you can pre-select your sample (instead of random door-to-door) and require multiple follow ups with the selected persons to reduce nonresponse bias.

You can also do a separate phone survey in addition to surveying the test participants. Questions like "Do you think you have had COVID?" and "How many times per week do you leave the house?". If the responses for the test participants vary significantly from the phone survey participants, you can try and weight your data accordingly.

Re:Preliminary test results suggest 21% of NYC residents hav... by ineedasername on Thursday April 23, 2020 @05:02PM (#22961380)
Sure, phone surveys for political purposes (presidential approval ratings etc.) have to deal with that all of the time. There are methods for estimating non-response impact. [0] One method of mitigating it that I've seen it to reach out again to non-responders. You then analyze their results to see how they differ from the baseline responders to estimate the non-responder population. If there's little/no difference, you can be fairly confident the risk of bias is low. It's called non-response follow up, and is a pretty common method.

There's also literature that suggests that you don't discard outlier values in the actual responders as they may help approximate the non-responder population, i.e., the non-outliers represent typical responders while outliers are more likely to represent non-responders [1]

[0]https://www.warc.com/content/paywall/article/jar/research_no...

[1] https://bmcmedresmethodol.biomedcentral.com/articles/10.1186...

Re:Preliminary test results suggest 21% of NYC residents hav... by ineedasername on Thursday April 23, 2020 @10:51PM (#22964160)
Wow, downvoted for providing factual information on how researchers mitigate non-response bias. Didn't think there was anything controversial there. I'm willing to accept the additional downvoted for this comment though.
Re:Preliminary test results suggest 21% of NYC residents hav... by austingulati on Friday April 24, 2020 @12:08PM (#22969574)
Not downvoted by me! Appreciate you sharing this information, it is helping me better understand how researchers mitigate non-response bias. Interesting stuff.
Re:Preliminary test results suggest 21% of NYC residents hav... by jonas21 on Thursday April 23, 2020 @03:29PM (#22960097)
Even with door to door knocking, you're going to introduce a bias -- toward people who are around to answer the door, which would tend to undersample essential workers.
Re:Preliminary test results suggest 21% of NYC residents hav... by AznHisoka on Thursday April 23, 2020 @03:36PM (#22960201)
Or people who will try to talk to you from their balcony rather than answering the door.