Déjà vu: Fast efficient probabilistic deduplication

Re:Déjà vu: Fast efficient probabilistic deduplication by bradknowles on Saturday July 15, 2017 @01:27AM (#14775155)

Could you use this to deduplicate files in a filesystem?

Could you use this as the deduplication method in ZFS?

[ Parent | on HN ]

Re:Déjà vu: Fast efficient probabilistic deduplication by f483 on Saturday July 15, 2017 @06:17AM (#14775954)

> Could you use this to deduplicate files in a filesystem?

Depends on what you mean and your constraints. Deduplicate file entries if you can live with a rare false positives, sure. A setup of 1million entrie limit with 1/1billion false positive chance will get ~80M mem usage.

If you want to check for duplicate files, probably not.

> Could you use this as the deduplication method in ZFS?

I would say no.

[ Parent | on HN ]