News for hackers. Stuff that matters.
Re:Base64 encoding and decoding at almost the speed of a mem... by _pmf_ on Wednesday November 06, 2019 @05:21AM (#21461203)
How does it kill compression?
Re:Base64 encoding and decoding at almost the speed of a mem... by londons_explore on Wednesday November 06, 2019 @05:46AM (#21461323)
Try it...

    wget http://mattmahoney.net/dc/enwik8.zip -O - | gunzip | head -c 1000000 | gzip -9 | wc -c
    355350
    wget http://mattmahoney.net/dc/enwik8.zip -O - | gunzip | head -c 1000000 | base64 | gzip -9 | wc -c
    528781
base64 is 48.8% larger after compression on english text, whereas it would only be 33.3% larger without compression. The reason is because compression finds and eliminates repeating patterns, but base64 can make the same input data look totally different depending on which of the 4 input alignments it has.
Re:Base64 encoding and decoding at almost the speed of a mem... by xxs on Wednesday November 06, 2019 @05:59AM (#21461374)
oddly enough lz/deflate are as ancient as base64. 16bit deflate is quite poor and slow, yet predominant.
Re:Base64 encoding and decoding at almost the speed of a mem... by alboy on Wednesday November 06, 2019 @05:50AM (#21461340)
Most compression algos operate on single-byte chunks of data and base64 encoding messes up the original byte alignment making the input appear more random than it is.
Re:Base64 encoding and decoding at almost the speed of a mem... by CamperBob2 on Wednesday November 06, 2019 @09:08AM (#21462703)
For one thing, it removes opportunities for more efficient content-aware compression elsewhere in the data path.
Re:Base64 encoding and decoding at almost the speed of a mem... by MaxBarraclough on Thursday November 07, 2019 @05:49PM (#21478114)
If we're going the content-aware route, can't the compression scheme just be Base64-aware?
Re:Base64 encoding and decoding at almost the speed of a mem... by CamperBob2 on Saturday November 09, 2019 @03:44AM (#21490412)
How exactly do you make a base64-aware image compressor, though?
Re:Base64 encoding and decoding at almost the speed of a mem... by MaxBarraclough on Monday November 11, 2019 @03:15PM (#21508423)
I don't follow. An image compressor is guaranteed not to be given Base64 input.

A general-purpose compression scheme can easily detect when its input is Base64, so I don't see why Base64 should be particularly hard to compress, in principle at least.