News for hackers. Stuff that matters.
Re:Show HN: A proof-of-concept FoundationDB based network bl... by stock_toaster on Saturday April 28, 2018 @05:11PM (#16949430)
You might give lz4 a try as well. ZFS uses it as a compression option, and the overhead is quite small (especially compared to zlib).
Re:Show HN: A proof-of-concept FoundationDB based network bl... by voidmain on Saturday April 28, 2018 @05:55PM (#16949615)
A sophisticated approach to block compression would be to build a dictionary using something like zstandard's "training mode" and a random sample of data blocks, store it (versioned) in FDB as well and reference the dictionary version from compressed blocks. That would get better compression with the relatively small blocks.