News for hackers. Stuff that matters.

A comment on Show HN: A proof-of-concept FoundationDB based network block device backend. Parent comment

Re:Show HN: A proof-of-concept FoundationDB based network bl... by dividuum on Saturday April 28, 2018 @03:13PM (#16949033)
Good question. I actually didn't test that at all. I mostly added that as I saw a few mostly empty blocks written and compression lowered the total space usage a bit.
[ Parent | on HN ]
Re:Show HN: A proof-of-concept FoundationDB based network bl... by stock_toaster on Saturday April 28, 2018 @05:11PM (#16949430)
You might give lz4 a try as well. ZFS uses it as a compression option, and the overhead is quite small (especially compared to zlib).
[ Parent | on HN ]
Re:Show HN: A proof-of-concept FoundationDB based network bl... by voidmain on Saturday April 28, 2018 @05:55PM (#16949615)
A sophisticated approach to block compression would be to build a dictionary using something like zstandard's "training mode" and a random sample of data blocks, store it (versioned) in FDB as well and reference the dictionary version from compressed blocks. That would get better compression with the relatively small blocks.
[ Parent | on HN ]