The continuous async replication feature you mention would be ideal, especially if it's setup so that I can use consul/etcd/... to automatically elect a "master". If you also added (1) a feature so that writes can be redirected to another instance, and (2) a feature so that I can direct SOME of my reads to a master (reads where I care about consistency), then you'd have a not too difficulty to implement - but also very powerful - distributed version of minio. Maybe something along those lines is what you are planning.

Many instances of minio makes a lot of sense to me, and it seems light weight enough (i.e. resources like memory) that it would work. The only deal-breaker feature which looked like it was missing was the ability to set a disk quota. In multi-tenant environments it's important to have resource limits for reliability reasons. I considered putting each minio instance in it's own ZFS volume to handle disk quotas, but I would have had to write a lot of glue code to make that work from a deployment perspective.

Great work on minio - it's the least painful to use S3 server implementation I've run across so far.