News for hackers. Stuff that matters.

A comment on Building a 30 PB storage cluster in the heart of SF.

Re:Building a 30 PB storage cluster in the heart of SF by Onavo on Wednesday October 01, 2025 @03:41PM (#45442369)
> We kept this obsessively simple instead of using MinIO or Ceph because we didn’t need any of the features they provided; it’s much, much simpler to debug a 200-line program than to debug Ceph, and we weren’t worried about redundancy or sharding. All our drives were formatted with XFS.

What do you plan to do if you start getting corruption and bitrot? The complexity of S3 comes with a lot of hard guarantees for data integrity.

[ Parent | on HN ]
Re:Building a 30 PB storage cluster in the heart of SF by g413n on Wednesday October 01, 2025 @04:33PM (#45443097)
our training stack doesn't make strong assumptions about data integrity, it's chill
[ Parent | on HN ]