Re:Hyperdrive – a novel file sharing network
by mindslight on Thursday December 10, 2015 @12:15PM
(#10711792)
(Not my project. I'm just commenting on the general techniques)
Rabin chunking should get you quick cross-file partial-file deduplication. So if you're already storing a file and then download a modified version (from someone else) with changed metadata in the header, only the differing data should be transferred/stored.
Rabin chunking should get you quick cross-file partial-file deduplication. So if you're already storing a file and then download a modified version (from someone else) with changed metadata in the header, only the differing data should be transferred/stored.