Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by 317070 on Saturday May 22, 2021 @06:16AM
(#27245155)
No, as I understand it, the network does not need to be transmitted. The decoder learns on the same stream as the encoder, so you do not need to transmit network parameters.
Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by ironSkillet on Saturday May 22, 2021 @08:15AM
(#27245774)
If I compress a large file at location A using this algorithm, and want to send it to location B, how does location B know how to decompress it?
Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by Yoric on Saturday May 22, 2021 @09:38AM
(#27246418)
Basically the same way your favorite codec knows how to decompress. It builds the predictor (in this case, a NN) from the known input. The predictor then assigns a symbol (or a series of symbols) to the next few bits - higher probabilities need fewer bits (I haven't checked for this specific technique, but there are techniques that don't even require an integer number of bits, e.g. range encoding).
Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by e12e on Saturday May 22, 2021 @08:32AM
(#27245891)
See https://news.ycombinator.com/item?id=27244810
As I understand it, when starting up (beginning of stream) - the network does not compress, then guesses the next byte at every turn determininstically from input so far.
Decoding can "read" the fist byte, then proceed to learn and guess the second byte and so on.
Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by clavigne on Saturday May 22, 2021 @08:31AM
(#27245885)
see the (current) top comment
https://news.ycombinator.com/item?id=27244810
it's a very clever use of symmetry.
Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by vidarh on Saturday May 22, 2021 @10:46AM
(#27247010)
It's clever, but pretty much "standard" in compression. Earliest I'm aware of that used symmetry in the encoder and decoder to prevent explicitly transferring the parameters this way was LZ78 (Lempel, Ziv; 1978), but there could well be predecessors I'm not aware of.
LZ78 used it "just" to build a dictionary, but the general idea of using symmetry is there.
(A fascinating variant on this is Semantic Dictionary Encoding, by Dr Michael Franz in his doctoral dissertation (1994), that used this symmetry to compress partial ASTs to cache partially generated code for runtime code generation)
Considering all other things like GPU, code quality etc. equal, the speed here mainly depends on the size of the network. Larger networks result in better performance (logarithmic), but also slower speeds (linear). There are some tricks like pruning to create significantly smaller networks with similar-ish performance, but good models are often large.
Unrelated to your question, what nobody mentions is that the decompression side also needs to complete network. With 57M and 187M parameters, those files are going to be quite large, > 100MB for sure. That completely annihilates the performance wins for 1 time transfers.