News for hackers. Stuff that matters.
Re:NNCP: Lossless Data Compression with Neural Networks by raphlinus on Saturday April 06, 2019 @09:53AM (#19590714)
And this is why Fabrice Bellard is so often cited as an example of a 10x programmer: his relatively simple LSTM-based neural net compressor is 10x faster than one of the more complex state of the art algorithms while delivering comparable compression ratios.
Re:NNCP: Lossless Data Compression with Neural Networks by mpais on Saturday April 06, 2019 @11:32AM (#19591218)
"his relatively simple LSTM-based neural net compressor is 10x faster than one of the more complex state of the art algorithms while delivering comparable compression ratios."

No, it's 10x faster than an experimental, completely performance-unoptimized LSTM model that is far from state-of-the-art on its own. From the paper:

"Regarding the speed, our results show that our implementation is much faster than lstm-compress[8] with a similar model and gain. The speed improvement comes from the more optimized matrix multiplication implementation and the larger batch size (16 instead of 1)."

Of most interest here is the library used, LibNC, which might allow us to speed up cmix a bit (even making it 2x faster would already help a lot in testing).

Disclaimer: I've worked on cmix with its author

Re:NNCP: Lossless Data Compression with Neural Networks by raphlinus on Saturday April 06, 2019 @11:40AM (#19591257)
Fair enough. cmix is awesome work, obviously, and I'm looking forward to the ideas becoming production-ready. Probably the most notable contribution by Fabrice Bellard is having something relatively simple.
Re:NNCP: Lossless Data Compression with Neural Networks by m0zg on Saturday April 06, 2019 @03:32PM (#19592561)
He's like 1000000x programmer if you consider the kinds of projects he successfully completes. An "average" programmer never does anything anywhere near as impactful as ffmpeg throughout the entirety of their career. And he has multiple of those under his belt.
Re:NNCP: Lossless Data Compression with Neural Networks by stabbles on Saturday April 06, 2019 @10:57AM (#19591031)
Well, to be fair, compression speed is not great: 17400 kB/s for gzip vs 2.75 kB/s for nncp (large network).
Re:NNCP: Lossless Data Compression with Neural Networks by tom_ on Saturday April 06, 2019 @11:33AM (#19591223)
And that's not all! - "The decompression speed is slower (1.5x slower for the LSTM model, 3x slower for the Transformer model)"...
Re:NNCP: Lossless Data Compression with Neural Networks by usefulcat on Saturday April 06, 2019 @09:35PM (#19594213)
It would be better to compare it to xz, which is significantly slower (and compresses better) than gzip.
Re:NNCP: Lossless Data Compression with Neural Networks by javierluraschi on Saturday April 06, 2019 @10:15AM (#19590818)
Where is the reference to the 10X improvement? As in, which benchmark are you comparing against? Thanks!
Re:NNCP: Lossless Data Compression with Neural Networks by beagle3 on Saturday April 06, 2019 @10:39AM (#19590950)
As in “one Fabrice Bellard is worth 10 or more average programmers”.
Re:NNCP: Lossless Data Compression with Neural Networks by javierluraschi on Saturday April 06, 2019 @11:05AM (#19591075)
I mean, regarding algorithm speed.

From the conclusion:

“We presented a practical implementation of an LSTM based lossless compressor. Although it is slow, its description is simple and the memory consumption is reasonable compared to compressors giving a similar compression ratio. It would greatly benefit from a dedicated hardware implementation.”

I only skimmed through the paper, but looks to me like this might be a 10X improvement over other LSTM compression methods, not state-of-the-art compression.

That said, I’m far from being an expert in compression and was hoping from someone in HN to explain how relevant this is in their discipline.

Re:NNCP: Lossless Data Compression with Neural Networks by stan_rogers on Saturday April 06, 2019 @11:14AM (#19591128)
The comment you are replying to was trying to explain the "10X programmer" idea. Nowhere was it claimed that the algorithm would have a 10X performance improvement.
Re:NNCP: Lossless Data Compression with Neural Networks by javierluraschi on Saturday April 06, 2019 @12:19PM (#19591487)
No, I meant this comment: "His relatively simple LSTM-based neural net compressor is 10x faster than one of the more complex state of the art algorithms"
Re:NNCP: Lossless Data Compression with Neural Networks by raphlinus on Saturday April 06, 2019 @11:13AM (#19591118)
I meant it as something of a joke (10x compression speed vs 10x the productivity of the average programmer), but also to summarize useful information from the paper.
Re:NNCP: Lossless Data Compression with Neural Networks by reubenmorais on Saturday April 06, 2019 @10:55AM (#19591018)
The paper linked in the article, table 3: https://bellard.org/nncp/nncp.pdf

I would add that lstm-compress from Byron Knoll, the author of CMIX, already achieved comparable compression ratios at 10x the speed, as is shown in the paper.

Re:NNCP: Lossless Data Compression with Neural Networks by tntn on Saturday April 06, 2019 @11:15AM (#19591130)
10x the speed of what? lstm-compress achieves a ratio of 1.64 bpb with a speed of 4.4 kB/s, Bellard's LSTM (small) achieves the same ratio with a speed of 41.7 kB/s.