News for hackers. Stuff that matters.
Re:OpenZL: An Open Source Format-Aware Compression Framework by yubblegum on Monday October 06, 2025 @06:25PM (#45496979)
Couldn't find in the paper a description of how the DAG itself is encoded. Any ideas?
Re:OpenZL: An Open Source Format-Aware Compression Framework by terrelln on Monday October 06, 2025 @06:28PM (#45497018)
We left it out of the paper because it is an implementation detail that is absolutely going to change as we evolve the format. This is the function that actually does it [0], but there really isn't anything special here. There are some bit-packing tricks to save some bits, but nothing crazy.

Down the line, we expect to improve this representation to shrink it further, which is important for small data. And to allow to move this representation, or parts of it, into a dictionary, for tiny data.

[0] https://github.com/facebook/openzl/blob/d1f05d0aa7b8d80627e5...

Re:OpenZL: An Open Source Format-Aware Compression Framework by yubblegum on Monday October 06, 2025 @06:33PM (#45497057)
Thanks! (Super cool idea btw.)