News for hackers. Stuff that matters.

A comment on Using machine learning to choose compression algorithms.

Re:Using machine learning to choose compression algorithms by proverbialbunny on Sunday December 08, 2019 @03:33PM (#21738102)
Is it possible to run data through a DNN, making sure the output is the same as the input (or close for lossless data), then take an autoencoder variant, record the 'compressed' data, then on the other end have the other half of the DNN to decompress it?

I'm pretty ignorant on the topic, so I get that that may be off, but if so, why wouldn't that be a valid solution to compressing data?

[ Parent | on HN ]
Re:Using machine learning to choose compression algorithms by d_burfoot on Monday December 09, 2019 @12:40AM (#21740628)
Sure, and you can add some correction bits to fix the lossy encoding of the autoencoder.

The problem is that you have to encode/send the DNN itself, otherwise your receiver won't know how to decode the data. If you are not smart, the added codelength of the DNN will likely blow away your savings. If you are smart, this leads to a whole formulation of machine learning called MDL:

https://en.wikipedia.org/wiki/Minimum_description_length

[ Parent | on HN ]
Re:Using machine learning to choose compression algorithms by visarga on Sunday December 08, 2019 @03:35PM (#21738115)
Autoencoders have lossy compression.
[ Parent | on HN ]
Re:Using machine learning to choose compression algorithms by kootenpv on Sunday December 08, 2019 @03:52PM (#21738227)
Indeed, but this is for lossless compression :)
[ Parent | on HN ]