Re:Show HN: A proof-of-concept FoundationDB based network bl...
by stock_toaster on Saturday April 28, 2018 @05:11PM
(#16949430)
You might give lz4 a try as well. ZFS uses it as a compression option, and the overhead is quite small (especially compared to zlib).
[ Parent
| on HN ]
Re:Show HN: A proof-of-concept FoundationDB based network bl...
by voidmain on Saturday April 28, 2018 @05:55PM
(#16949615)
A sophisticated approach to block compression would be to build a dictionary using something like zstandard's "training mode" and a random sample of data blocks, store it (versioned) in FDB as well and reference the dictionary version from compressed blocks. That would get better compression with the relatively small blocks.