Re:Ask HN: How would you store 10PB of data for your startup...
by monstrado on Friday April 23, 2021 @04:39PM
(#26918960)
MinIO is an option as well and would allow you to transition from testing in S3 to your own MinIO cluster seamlessly.
[ Parent
| on HN ]
Re:Ask HN: How would you store 10PB of data for your startup...
by goliatone on Friday April 23, 2021 @11:06PM
(#26921916)
I wonder if anyone can comment if they have experience running minio at scale. It would be a pleasant surprise if a “simple” minio cluster could handle such workload
Re:Ask HN: How would you store 10PB of data for your startup...
by dividedbyzero on Friday April 23, 2021 @05:52PM
(#26919647)
Does it scale that far?
Re:Ask HN: How would you store 10PB of data for your startup...
by pas on Friday April 23, 2021 @07:16PM
(#26920380)
Likely not that "gracefully". But Ceph absolutely does, and has an S3 gateway.
Re:Ask HN: How would you store 10PB of data for your startup...
by tempest_ on Friday April 23, 2021 @08:04PM
(#26920734)
Using Ceph like S3 could be a bit tricky if all of that 10pb is very small files.
Redhat did an interesting series of blog posts about Ceph and getting it to 1 billion objects
https://www.redhat.com/en/blog/scaling-ceph-billion-objects-...