News for hackers. Stuff that matters.
Re:NNCP: Lossless Data Compression with Neural Networks by javierluraschi on Saturday April 06, 2019 @10:15AM (#19590818)
Where is the reference to the 10X improvement? As in, which benchmark are you comparing against? Thanks!
Re:NNCP: Lossless Data Compression with Neural Networks by beagle3 on Saturday April 06, 2019 @10:39AM (#19590950)
As in “one Fabrice Bellard is worth 10 or more average programmers”.
Re:NNCP: Lossless Data Compression with Neural Networks by javierluraschi on Saturday April 06, 2019 @11:05AM (#19591075)
I mean, regarding algorithm speed.

From the conclusion:

“We presented a practical implementation of an LSTM based lossless compressor. Although it is slow, its description is simple and the memory consumption is reasonable compared to compressors giving a similar compression ratio. It would greatly benefit from a dedicated hardware implementation.”

I only skimmed through the paper, but looks to me like this might be a 10X improvement over other LSTM compression methods, not state-of-the-art compression.

That said, I’m far from being an expert in compression and was hoping from someone in HN to explain how relevant this is in their discipline.

Re:NNCP: Lossless Data Compression with Neural Networks by stan_rogers on Saturday April 06, 2019 @11:14AM (#19591128)
The comment you are replying to was trying to explain the "10X programmer" idea. Nowhere was it claimed that the algorithm would have a 10X performance improvement.
Re:NNCP: Lossless Data Compression with Neural Networks by javierluraschi on Saturday April 06, 2019 @12:19PM (#19591487)
No, I meant this comment: "His relatively simple LSTM-based neural net compressor is 10x faster than one of the more complex state of the art algorithms"
Re:NNCP: Lossless Data Compression with Neural Networks by raphlinus on Saturday April 06, 2019 @11:13AM (#19591118)
I meant it as something of a joke (10x compression speed vs 10x the productivity of the average programmer), but also to summarize useful information from the paper.
Re:NNCP: Lossless Data Compression with Neural Networks by reubenmorais on Saturday April 06, 2019 @10:55AM (#19591018)
The paper linked in the article, table 3: https://bellard.org/nncp/nncp.pdf

I would add that lstm-compress from Byron Knoll, the author of CMIX, already achieved comparable compression ratios at 10x the speed, as is shown in the paper.

Re:NNCP: Lossless Data Compression with Neural Networks by tntn on Saturday April 06, 2019 @11:15AM (#19591130)
10x the speed of what? lstm-compress achieves a ratio of 1.64 bpb with a speed of 4.4 kB/s, Bellard's LSTM (small) achieves the same ratio with a speed of 41.7 kB/s.