Re:Bitpacking and Compression of Sparse Datasets
by delhanty on Sunday January 08, 2017 @12:11AM
(#13348389)
Your comment, particularly the degradation in byte alignment leading to worse subsequent compression, reminds me of the following Hacker News comment by NelsonMinar from then end of November 2016:
>It's crucial to evaluate encoding space usage in the context of compression. For instance gzip(base16(data)) is often smaller than gzip(base64(data)) for practical data. Even though base64 is more efficient than base16, it breaks up data across byte boundaries which then makes gzip significantly less efficient.
https://news.ycombinator.com/item?id=13049894
>It's crucial to evaluate encoding space usage in the context of compression. For instance gzip(base16(data)) is often smaller than gzip(base64(data)) for practical data. Even though base64 is more efficient than base16, it breaks up data across byte boundaries which then makes gzip significantly less efficient.