News for hackers. Stuff that matters.

A comment on A Distributed File System in Go Cut Average Metadata Memory Usage to 100 Bytes. Parent comment

Re:A Distributed File System in Go Cut Average Metadata Memo... by amluto on Friday February 23, 2024 @10:41AM (#39481938)
Does JuiceFS allow the user to specify the hash of a file when uploaded? And then to read that hash back later?

Otherwise there’s no end-to-end integrity check.

[ Parent | on HN ]
Re:A Distributed File System in Go Cut Average Metadata Memo... by markhahn on Friday February 23, 2024 @10:58AM (#39482207)
surely you mean that the FS should calculate the hash on file creation/update, not take some random value from the user. but I agree that a FS that maintains file-content hash should allow clients to query it.
[ Parent | on HN ]
Re:A Distributed File System in Go Cut Average Metadata Memo... by amluto on Friday February 23, 2024 @11:43AM (#39482878)
No, the FS should verify the hash on creation/update. Otherwise corruption during creation/update would just cause the hash to match the corrupted data.
[ Parent | on HN ]
Re:A Distributed File System in Go Cut Average Metadata Memo... by daviesliu on Sunday February 25, 2024 @05:51AM (#39499698)
The S3 API allow user to specify the hash of content as HTTP header, it will be verified by the JuiceFS gateway and persisted into JuiceFS as ETag.

With POSIX API or HDFS, there is no such API to do that, unfortunately.

[ Parent | on HN ]