News for hackers. Stuff that matters.

A comment on Hyperdrive – a novel file sharing network. Parent comment

Re:Hyperdrive – a novel file sharing network by zbjornson on Wednesday December 09, 2015 @10:39PM (#10708720)
Pardon my ignorance, but I'm still not quite understanding the applications and benefits. Distributed file hosting? High-concurrency reads? Torrent-like? "file sharing network" is a bit ambiguous to me.

(I seem to have broke the demo with a 100 MB file, btw.)

[ Parent | on HN ]
Re:Hyperdrive – a novel file sharing network by mindslight on Thursday December 10, 2015 @12:15PM (#10711792)
(Not my project. I'm just commenting on the general techniques)

Rabin chunking should get you quick cross-file partial-file deduplication. So if you're already storing a file and then download a modified version (from someone else) with changed metadata in the header, only the differing data should be transferred/stored.

[ Parent | on HN ]