Re:MinIO: A Bare Metal Drop-In for AWS S3
by gpapilion on Tuesday August 10, 2021 @01:24PM
(#28131620)
HDFS doesn't really work as a normal filesystem. I think some other commenters pointed out the challenges with FUSE.
If I recall correctly there isn't really a way to modify an existing file via HDFS, so you'd have to copy/edit/replace. Append used to be an issue, but that got sorted out a few years back.
Erasure coding is available in the latest versions. Which helps with replication costs.
I think, HDFS may just be a simpler setup than other solutions. (which is to say its not all that simple, but easier than some other choices). And I wouldn't use HDFS as a replacement for block storage, which is something I've seen done with Ceph.
Re:MinIO: A Bare Metal Drop-In for AWS S3
by tinco on Tuesday August 10, 2021 @05:36PM
(#28134956)
Thanks, we actually use Ceph as a straight up filesystem, that gets mounted on linux machines and then exposed to our windows based processing nodes (they are human operated) over SMB. I think that explains why HDFS is not a good fit for us.
If I recall correctly there isn't really a way to modify an existing file via HDFS, so you'd have to copy/edit/replace. Append used to be an issue, but that got sorted out a few years back.
Erasure coding is available in the latest versions. Which helps with replication costs.
I think, HDFS may just be a simpler setup than other solutions. (which is to say its not all that simple, but easier than some other choices). And I wouldn't use HDFS as a replacement for block storage, which is something I've seen done with Ceph.