Re:pigz: A parallel implementation of gzip for multi-core ma...
by elmolino89 on Tuesday October 18, 2022 @08:12AM
(#33246040)
bgzf has a clear advantage when it comes to sorted BED/VCF/GTF formats, especially if one does index these.
But frankly I have no idea if it may improve IO times for the fastq files when read by say bwa or another mapper. Do you have any experience with that?
I have seen some mapping times improvement using fastqs with clustered reads (by clumpify).
[ Parent
| on HN ]
Re:pigz: A parallel implementation of gzip for multi-core ma...
by jakobnissen on Tuesday October 18, 2022 @09:45AM
(#33247025)
Yes, bgzf can be much faster in practice. Both because the underlying gzip implementation can be simplified (see e.g. libdeflate), and because it can be more effectively parallelised. It doesn't matter if it's IO-bound of course, but compression rarely is.