Most importantly, I think a system like this must be non-profit and freely available for global usage. Privacy issues can be solved with careful ETL process. Ideally a single global platform...although political realities may force multiple instances.
Regardless of implementation details - this seems to fit the YC Research charter in terms of medium/long horizons and big 'change the world' payoffs.
Today a physician suspecting novel association e.g. Adverse event particular to a co-morbid condition. Usually has to wait multiple weeks for Ordering data from US government, Developing SAS scripts and finally conducting the study. Wrth our stack the underlying question can be answered within seconds. With enough legal permissions we can modify it to deliver only the required data, for further statistical analysis. Sure such a system might not be completely public, but there is nothing that prevents us from giving access to say all board certified physicians globally.
Not to put a spanner in your works, as I agree your goal is worthy of a lot of effort from the general public. But, how exactly did you get access to this data, and is it legal?
Additionally, from my point of view, even if my medical "records" were de-personalized and made anonymous, I still want to have full control over who/what get's access to it.
Regarding your point second point, the patient ownership of the data is not very well understood legally. Since most of it is transactional in nature. E.g. consider the Sorrell vs IMS Health judgement by the Supreme Court few years ago. Following paper in Harvard Journal of Law and Technology gives a good overview of the issues.
http://jolt.law.harvard.edu/articles/pdf/v25/25HarvJLTech69....
http://research.microsoft.com/pubs/74339/dwork_tamc.pdf
Building a database which can be locked down to differentially private primitives (differential privacy composes) would allow researchers to partially unlock this data while ensuring that your medical records are private.
As long as we choose epsilon (the privacy parameter) sufficiently low there is then no ethical need to ask a bunch of fickle data points for permission.
Differential Privacy cannot be directly applied since the underlying assumptions are too strong. An important consideration is that the error/noise added is independent of the answer. Which means that the system becomes unusable for almost all queries other than general trends.
By restricting the query structure, we no longer need large amount of noise. Privacy of hospitals and providers is also very important and cannot be encoded in the Differential Privacy framework. Again this is still a hotly debated issue. But even the most vocal supporters of differential privacy agree that it might not be directly applicable for healthcare domain.
Following are some of the paper that discuss this:
http://www.openu.ac.il/personal_sites/tamirtassa/download/co...
http://www.jetlaw.org/wp-content/uploads/2014/06/Bambauer_Fi...
I.e., if you don't have differential privacy, then by definition there is a de-anonymizing query and you can get PII out.
Privacy of hospitals/providers is a separate issue, and yeah, it's pretty clear that differential privacy doesn't work for them. Thanks for the links, I'll check them out.
Edit: after reading your second article, it's deeply misleading. They assume that to compute a mean, one must compute 2 queries - sum(x) and len(x), each of which must be differentially private. But that's totally wrong! You can in fact run the query mean(x) + noise, and this last query itself can be differentially private.
The article also notes that queries on small data sets require more noise to be differentially private, which is totally true, and obvious.
This also, however, ignores the fact that most statistical inference drawn from such queries will be nonsense even without differential privacy. See, e.g., this article for an example of why: https://www.chrisstucchio.com/blog/2015/ab_testing_segments_...
This is a very bad critique of differential privacy.
Regarding
"""This also, however, ignores the fact that most statistical inference drawn from such queries will be nonsense even without differential privacy."""
This is not true. Just because the number returned by a count query on a very large dataset (~100 Million visits) is very small (~100) does not automatically means that the result is nonsense or can be disregarded as error. Doing that requires understanding the query and a hypothesis with good prior on expected outcome. E.g. intersection of two rare diseases. Where you would otherwise expect it to be very small, but there might be an underlying genetic reason / physiological process which might lead to higher prevalence. Or a group of hospitals using tainted batch of medicines leading to unexplained increased mortality.
Consider this paper where there were only 1000 cases (only 248 strokes) per 1.6 Million patients (even larger if you consider the entire 20 Million patients present in the data). However in spite of the small number the authors showed that the increase was statistically significant by comparing with same period a year later.
http://www.nejm.org/doi/full/10.1056/NEJMoa1311485
Again I am not denying what you wrote in the blogpost. But in medicine and the analyses for which such databases are used, the investigators have access to very good priors.
Here is the link http://www.computationalhealthcare.com
We have access to almost 130 Million de-identified medical records from approximately 36 Million patients (~10% of US population) this includes all Inpatient, ED, Ambulatory Surgery records between 2006-2011 from California. To put simply if you lived in California and went to a hospital there is 95.9% chance that we have your data. This data has been available for quite some time but its use has been hindered due to lack of good software. The data has led to significant research, e.g. my collaborator (not me) published a paper showing risk of strokes following pregnancies in New England Journal of Medicine last year.
At Cornell Tech & Weill Cornell Medical College, we have developed a Search and Aggregation engine that will revolutionize how researchers and physicians use this data. Imagine your mother with Leukemia in Remission just got admitted for Pneumonia. With our software, the Physician will be quickly able to asses likelihood of this occurring and rule out any confounding adverse events. Or consider that there is a rare combination of diagnosis e.g. Graves Disease and Clotting disorder that is indicative of a unique genetic mutation likely to offer novel insight into disease process. With our software questions like these can be answered within second, Today & Right now.
The Data, Legal structure and fully functional prototype are available right now. We were counting on support from AHRQ, but sadly the agency has run into trouble due significant budget cuts.