News for hackers. Stuff that matters.
Re:What to do when you don’t trust your data anymore by 6gvONxR4sf7o on Friday January 31, 2020 @11:10AM (#22202762)
It's amazing how common it is to see something that doesn't make sense in the data and instead of deciding to investigate, they just move on. I'm guilty too sometimes. But almost every time I dig, I find something that invalidates it. Maybe an ETL somewhere is putting an empty string instead of NULL in particular cases. Maybe my join is duplicating certain rows. Maybe you got a table (or spreadsheet) that doesn't make sense. It all violates the assumptions of an analysis, but collectively we just say "can't eliminate all bugs." I hope the future is better.
Re:What to do when you don’t trust your data anymore by perl4ever on Friday January 31, 2020 @06:29PM (#22207076)
Some places, they do QA, and when they find something in a random sample, they fix that instance and continue, rather than looking for all the places it happened and why.