Re:pigz: A parallel implementation of gzip for multi-core ma...
by hoseja on Tuesday October 18, 2022 @06:32AM
(#33245275)
Because scientists are woefully undereducated on the finer points of computer science. Does it (eventually) produce publishable papers? is all they care about.
Re:pigz: A parallel implementation of gzip for multi-core ma...
by exyi on Tuesday October 18, 2022 @07:33AM
(#33245743)
It was probably fasta of fastq format, which is pretty much line separated strings of the DNA letters. BSON won't help with that. You could try to squeeze the each of the 4 letters into 2 bits, but they use few more letters to indicate "unknown", so it's not that easy. And even the simplest compression algorithms just do that for you.
And text is easier to work with in any hacked up script.
There are some binary formats (BAM, I think), but people often prefer the text format anyway. When compressed, the size is pretty much the same
Re:pigz: A parallel implementation of gzip for multi-core ma...
by nextaccountic on Tuesday October 18, 2022 @12:48PM
(#33250308)
> And even the simplest compression algorithms just do that for you.
They do, but you need to uncompress to read the N-th letter
Why not use some binary format like BSON? Compressing with gzip works but then you can't query it compressed