Re:Game Boy Wordle clone: How to compress 12972 five-letter ...
by quicktwo on Monday February 21, 2022 @10:02PM
(#30423578)
You can beat 15,412 bytes with a much simpler decompression algorithm than xz. Using variable length encoding on a tail-sorted delta-offset array gets down to 13,180 bytes.
Neither competes with RoadRoller (which gets down to around 12,200 and includes the code for decoding), but that takes forever to decompress and uses a ton of memory so certainly not applicable for this application.
See my other comments in this thread if you have interest!
Original word list: https://raw.githubusercontent.com/arpruss/gb-fiver/main/comp...
2000 bytes smaller without any optimizations (this even includes the newlines), though now I guess the question is what the size of the xz decompressor is since a few KB actually matter (not like on a regular computer).For some reason bzip2 gets it only to 36K, even worse than gzip (32K) and zstd (29K).
Update: Counter-intuitively, stripping the newlines (... | tr -d \\n | ...) results in a higher compressed size with xz. It's surprisingly hard to find a minimal xz decompressor, it doesn't seem as though anyone bothers with this stuff. The Hutter Prize includes the decompressor so that was my first stop, but none of the contestants submitted that (I assume they predate xz).
Update#2: Found at least one measurement of xz decompressor at 36K, but it seems to me like this is the general-purpose utility and includes the compressor, help output, etc. http://mattmahoney.net/dc/text.html