News for hackers. Stuff that matters.
Re:Florida scientist fired for refusing to manipulate Covid-... by AnthonyMouse on Tuesday May 19, 2020 @02:26PM (#23238275)
There is a third option here. Don't publish data until it's collected. This means you don't publish data for deaths on a date until all the deaths on that date have been reported. Then you don't have to retroactively update anything because you didn't publish numbers before they're finalized.

This option is clearly better than publishing date of report as date of death, because doing that is inaccurate. The data is really from events a week or more ago when you think it's from today, and you can get clusters like this when many reports come in at once.

The disadvantage is that you then have newer data that isn't being published because it isn't completely in yet. But if you are going to publish incomplete data then it should still be date of death and the dates without full reporting should be annotated so that people understand that they're looking at incomplete data -- which they are, so it's very important for that to be understood.

Re:Florida scientist fired for refusing to manipulate Covid-... by vharuck on Tuesday May 19, 2020 @03:43PM (#23239252)
It's a sliding scale between immediacy and accuracy, with a very high upper length of time for perfect accuracy. I work with death record statistics for a state government, and our reports use data from 2 years ago. A big reason is gathering reports for residents who died in other states.

I know the CDC has a good system for sharing infectious disease diagnoses (https://wwwn.cdc.gov/nndss). I don't know if there's a similar rapid-collection system for death records. We usually don't need to track deaths with this urgency.

A possible compromise is to model a delayed reporting factor. Not sure if there could be a good enough model, though.

Re:Florida scientist fired for refusing to manipulate Covid-... by rovolo on Tuesday May 19, 2020 @03:08PM (#23238815)
> don't publish data for deaths on a date until all the deaths on that date have been reported

Based on the animation I linked, it takes around 15 weeks for an apparent decrease in pneumonia deaths to rise to a normal number. 15 weeks ago was Feb 4. (Granted, pneumonia death data collection in normal times will be different than covid-specific data now). The numbers do get more accurate with time though. There is a trade-off between accuracy and timeliness.