Re:A poor entropy coding result in Zstandard
by theandrewbailey on Monday October 02, 2023 @10:12AM
(#37738406)
> So the theoretical minimum is ~1.2 MB. Zstd is only acheiving ~1.4 MB, which is nowhere near the theoretical limit.
> Interestingly, 7-zip compresses the data much better than zstd, reaching a size of 1,298,590 B.
This isn't surprising to me. Zstd is designed for real time compression at I/O speeds (~1 GB/s), so it probably would leave some entropy in exchange for speed, but LZMA (7zip default) is designed for absolute best compression ratio.
> Interestingly, 7-zip compresses the data much better than zstd, reaching a size of 1,298,590 B.
This isn't surprising to me. Zstd is designed for real time compression at I/O speeds (~1 GB/s), so it probably would leave some entropy in exchange for speed, but LZMA (7zip default) is designed for absolute best compression ratio.