News for hackers. Stuff that matters.
Re:Open Sourcing My Personal Medical Record by kharms on Thursday May 04, 2017 @02:27PM (#14267382)
>We should be dumping these whole datasets into machine learning and having computers give us potential links to explore.

You're describing P-value hacking, thus named because hack scientists use this technique to publish papers about nonsense.

Re:Open Sourcing My Personal Medical Record by highd on Thursday May 04, 2017 @03:47PM (#14268109)
You basically just need to reduce the P-value you need to claim significance (0.1 to 0.001 or less) to account for the probability of finding those correlations even in noise. This is part of why particle physics has such high standards - you can find a lot of things in the TBs of data CERN generates.

See for example: https://en.wikipedia.org/wiki/Genome-wide_association_study

There's a figure in there depicting associations with P-values of 1e-8: https://en.wikipedia.org/wiki/Genome-wide_association_study#...