News for hackers. Stuff that matters.
Re:Wikipedia-IPFS: An exploration to host Wikipedia in IPFS by toomuchtodo on Sunday May 10, 2020 @02:51PM (#23135005)
It’s a shame modern file systems don’t generate and store common hash algorithms as file metadata.
Re:Wikipedia-IPFS: An exploration to host Wikipedia in IPFS by imhoguy on Sunday May 10, 2020 @03:54PM (#23135566)
Some file systems like ZFS, btrfs support checksum on block level. However file level checksum is not trivial with random write file system. Imagine 1 byte is changed in 20GB file - that would require full file scan to update the checksum. Recalculating a file checksum from all block checksums could be a solution however far from any standard.
Re:Wikipedia-IPFS: An exploration to host Wikipedia in IPFS by brokenmachine on Sunday May 10, 2020 @09:58PM (#23138171)
Wouldn't a merkle tree, like torrents or dat use, avoid this? You'd only have to generate a hash of the changed chunk, then it's really low cost to update the root hash.

https://datprotocol.github.io/book/ch01-02-merkle-tree.html

Re:Wikipedia-IPFS: An exploration to host Wikipedia in IPFS by imhoguy on Monday May 11, 2020 @08:03AM (#23141197)
Yep, my last sentence covered also Markle tree.