How Microsoft computer scientists and researchers are working to 'solve' cancer

Re:How Microsoft computer scientists and researchers are wor... by patall on Wednesday September 21, 2016 @10:17AM (#12548329)

There are. Data types are pretty much standardized on the basic level (eg. fastq, bam, bed, vcf etc.). Else, most is done by the big science consortia like TCGA, ICGC & PanCancer. And obviously the big databases from NCBI and EBI-EMBL. Trials are standardized by EMA and FDA.

[ Parent | on HN ]

Re:How Microsoft computer scientists and researchers are wor... by dekhn on Wednesday September 21, 2016 @10:25AM (#12548406)

FASTQ, BAM, BED, VCF, etc are all standardized in terms of file format, but the semantics of the content are not. Harmonizing data across multiple semantics is still a very challenging problem that few people take seriously.

[ Parent | on HN ]

Re:How Microsoft computer scientists and researchers are wor... by patall on Wednesday September 21, 2016 @10:32AM (#12548466)

Like the good old "chr1" == "1" problem? Well, yeah that’s true. But still I think we have those consortia in place, why do we need another authority. They just have to set (and somewhat enforce) the rules.

[ Parent | on HN ]

Re:How Microsoft computer scientists and researchers are wor... by dekhn on Wednesday September 21, 2016 @11:52AM (#12549316)

It goes far beyond "chr1" = "1". There are many other fields with next-to-impossible to interpret data. The consortia have, at best, advisory roles, they cannot enforce any rules. And given an extensible file format, third parties are going to extend it with poorly defined/standardized data. Then the consumers have to harmonize the data. Data harmonization (or cleaning) it typically one of the bottleneck steps in applying machine learning at scale.

[ Parent | on HN ]