Re:NNCP: Lossless Data Compression with Neural Networks (2019)
by AtlasBarfed on Saturday May 22, 2021 @01:18PM
(#27248484)
Hm, there are likely multiple large files that exist that are very common (DLLs, versions of shared libs, etc etc etc) that might be frequently compressed (SCP, stored in SSDs, etc) since disk space isn't free yet last I checked.
Each file is really its own special theoretical multivariate space to find the best theoretical compression ratio. Usually finding the optimal or near-optimal compression is the hardest computationally.
And it's basically a perfectly cacheable computation if you have some sort of database mapping algorith + parameters to apply to a given file.
Decompression usually isn't as big a deal for performance.
Anyway an online database for a service that has actively tried to take a known common file (match on filesize, hash, etc) down to its best compressed representation so 1) we don't have to do it every time.
Cloud providers would presumably love these, since S3 probably has millions copies of many files and would 1) get to compress the file and 2) get to share its representation across all instances.
Each file is really its own special theoretical multivariate space to find the best theoretical compression ratio. Usually finding the optimal or near-optimal compression is the hardest computationally.
And it's basically a perfectly cacheable computation if you have some sort of database mapping algorith + parameters to apply to a given file.
Decompression usually isn't as big a deal for performance.
Anyway an online database for a service that has actively tried to take a known common file (match on filesize, hash, etc) down to its best compressed representation so 1) we don't have to do it every time.
Cloud providers would presumably love these, since S3 probably has millions copies of many files and would 1) get to compress the file and 2) get to share its representation across all instances.