News for hackers. Stuff that matters.
Re:What is the space overhead of Base64 encoding? by NelsonMinar on Wednesday January 30, 2019 @08:56PM (#19041579)
I wonder if gzip has improved? 15 years ago when I tested this, it was more efficient to gzip base16 encoded data than base64 encoded data. (At least, the English dictionary.) I assume that was because the 3:4 encoding broke up patterns in the source text and messed with the compressor.

But I just tested this again and it's not true anymore.

  $ wc -c american-english  
  971578 american-english
  
  $ gzip < american-english | wc -c  
  259977

  $ base64 < american-english | gzip | wc -c  
  429263

  $ hexdump -e '"%x"' < american-english | gzip | wc -c  
  478411
Re:What is the space overhead of Base64 encoding? by kbaker on Wednesday January 30, 2019 @10:13PM (#19041978)
Actually, it is still true. hexdump has some confusing format options... what you were using was converting it to little-endian first before printing the hex representation, which really messed with gzip. Try this:

    $ hexdump -e '"%x"' < american-english | gzip | wc -c
    463871
    $ hexdump -v -e '/1 "%02x"' < american-english | gzip | wc -c
    302515
    $ base64 < american-english | gzip | wc -c
    415737
Re:What is the space overhead of Base64 encoding? by eadmund on Thursday January 31, 2019 @10:04AM (#19045162)
> converting it to little-endian first

Ack, is there anything little-endian doesn't ruin?

/me misses 68000 assembler …

Re:What is the space overhead of Base64 encoding? by NelsonMinar on Thursday January 31, 2019 @11:35AM (#19046032)
Thanks! That'll teach me to cut and paste from StackExchange. Also why I got different results last time I looked at this; was using a different base 16 encoder.