Re:The Desperate Quest for Genomic Compression Algorithms
by shusson on Wednesday August 22, 2018 @05:03PM
(#17822174)
> Thus, the pre-filter would find a good fit against the reference copy, compute a diff that's sparse, and compress that.
Yeah that already happens [1]. But before you can compute the diff, you have to read the genome with a level of confidence, which generates a lot of data.
Re:The Desperate Quest for Genomic Compression Algorithms
by rishav_sharan on Thursday August 23, 2018 @02:27AM
(#17825046)
I don't understand this field but wouldn't creating a reference model be an ethical and cultural nightmare? I would assume that researchers will be forced to make a very diluted version of this reference model which cannot be tied to any race, which at the end of the day may not be much useful.
Re:The Desperate Quest for Genomic Compression Algorithms
by shusson on Thursday August 23, 2018 @08:13AM
(#17826545)
On average there is only a 0.6% difference between human genomes [1]. And the reference genome already exists and is constantly being updated [2].
[1] https://en.wikipedia.org/wiki/Human_genetic_variation [2] https://en.wikipedia.org/wiki/Reference_genome
However the most obvious starting idea to me is that most humans will have mostly the same DNA, with some small 'configuration' changes and maybe variations in some smaller sections.
Thus, the pre-filter would find a good fit against the reference copy, compute a diff that's sparse, and compress that.
Someone more versed in the field could probably provide a better estimate of how to address that. Taking the 3.25Billion bases (addresses) mentioned in another post, 32 bits of storage is sufficient for an unsigned index. The reference variation should also be stored, but I don't know how many variations each section might have.
Probably after a sufficiently large collection is amassed the collections could be better sorted such that the most common have a prefix which is more compressible assigned to them (and they should also be tested first by the compressor).