News for hackers. Stuff that matters.
Re:Shakespeare.txt.jpg: A JPEG compression experiment by DanBC on Monday June 17, 2013 @05:51PM (#5896209)
So, this is a fun bit of tinkering and all, but I'd be more interested to see just how small HN could losslessly compress the Gutenberg plain text UTF8 file of Romeo and Juliet.

(http://www.gutenberg.org/cache/epub/1112/pg1112.txt)

Especially if you're using interesting techniques to build the dictionary.

EDIT: I got it down from 178983 to 44830 bytes, which isn't as impressive as I was hoping for.

EDIT2: Currently at 43,666 bytes which is a bit better.

EDIT3: 40,125 bytes.

Re:Shakespeare.txt.jpg: A JPEG compression experiment by abecedarius on Monday June 17, 2013 @08:51PM (#5896910)
You'd want to count the size of the decoder, like http://en.wikipedia.org/wiki/Hutter_Prize
Re:Shakespeare.txt.jpg: A JPEG compression experiment by csense on Tuesday June 18, 2013 @01:52AM (#5897785)
Yeah. You really need this caveat to avoid the loophole where you can compress anything to zero bytes by making the desired output a literal in your decompressor (which just ignores its input and prints the hard-coded literal).

This could be considered a real-world application of "hello world" programs: Winning compression prize contests by blatantly abusing the rules.

Re:Shakespeare.txt.jpg: A JPEG compression experiment by DanBC on Tuesday June 18, 2013 @03:19AM (#5898009)
Well, yes. Except the source text is tiny, so unfortunately all the software is pretty big in comparison.

I love the Hutter Prize, and the various compression benchmark sites.

Re:Shakespeare.txt.jpg: A JPEG compression experiment by abecedarius on Tuesday June 18, 2013 @03:33PM (#5901461)
It's just pretty unconstrained without that kind of rule. And a decompressor that doesn't have to be fast can be tiny.
Re:Shakespeare.txt.jpg: A JPEG compression experiment by voltagex_ on Monday June 17, 2013 @07:41PM (#5896677)
Now there's a code golf challenge I can work with. Mind posting some of your techniques?
Re:Shakespeare.txt.jpg: A JPEG compression experiment by DanBC on Tuesday June 18, 2013 @03:21AM (#5898016)
Oh! At the moment I'm just using existing software (PAQ8 variants, and WinRK).

WinRK is best so far. It's not free. It has a 30 day trial. (http://winrk.com/)

Re:Shakespeare.txt.jpg: A JPEG compression experiment by csense on Tuesday June 18, 2013 @01:47AM (#5897775)
General-purpose compressors for comparison:

   gzip      : 66513
   gzip  -c9 : 66099
   bzip2 -c9 : 53619
   xz    -c9 : 59572
Interestingly, bzip2 is the best. Often xz is better, but apparently not for this data.