News for hackers. Stuff that matters.
Re:Playing with Docker 1.13 and Minio by general_ai on Saturday January 21, 2017 @07:08PM (#13452831)
Does anyone have large (PB-scale) deployments of Minio on premises? We have a need to store hundreds of terabytes of data, and we don't need a filesystem per se, so I was wondering how robust it is.
Re:Playing with Docker 1.13 and Minio by y4m4b4 on Sunday January 22, 2017 @01:44AM (#13453991)
With 16 x 8TB drives, we are talking about 64TB usable space after erasure coding. Minio is designed to serve the needs of a single tenant - 64TB is usually much bigger than a single tenant's needs. However you often see 100s of thousands of such tenants in a cloud like environment.

Making a single large PB sized volume where the disks and nodes are managed like a blackbox by the filesystem is quite scary to us at Minio. Any crash means we blew up all the tenants at once. 1000s of individual Minio tenants means, we know when we add the million'th minio instance, it is not any more complex than the first instance of Minio.

Provisioning with k8s [1], mesos [2] or other external orchestration tools is better than Minio's own resource management system. When it comes to the applications, objects are just represented as URLs. Some data sitting on Amazon S3 and some on Minio makes no difference to the application.

We are also planning to introduce a namespace layer to work on top to combine these individual tenants into single namespace. This would allow you to transparently add new clusters as you scale without loosing the ability to have a single namespace.

We hang out at slack [3], please reach out if you have further questions.

[1] - https://github.com/kubernetes/charts/tree/master/stable/mini... [2] - https://github.com/dcos/examples/tree/master/1.8/minio [3] - https://slack.minio.io

Re:Playing with Docker 1.13 and Minio by alrs on Saturday January 21, 2017 @08:42PM (#13453103)
Physically impossible to run a PB-scale Minio cluster: it's limited to 16 hard drives. See my comment above.
Re:Playing with Docker 1.13 and Minio by general_ai on Saturday January 21, 2017 @09:42PM (#13453296)
Good to know. Looks like more of a PR initiative then. Some of my servers have more drives than that. :-)
Re:Playing with Docker 1.13 and Minio by unlocksmith on Sunday January 22, 2017 @05:09PM (#13457888)
Minio shards data across 16 drives. If you have more drives, you would run multiple instances for each set of 16 drives. I would generally recommend against denser storage servers. When it goes down, all the drives go offline with it.