I'm super confused how anyone could recommend Minio for the foundation of a production system as well. I've understood it to be primarily for development purposes and have used it for some local mock tests for S3 compatibility behind a firewall or to simulate error test cases more reliably in my applications.

Even Ceph has its warts for distributed object storage as does basically... anything in existence worth considering that's OSS-ish (GlusterFS, HDFS, Lustre) but comparing Minio to ZFS is confusing given how drastically different in purpose, functionality, and engineering hardening has happened between the two projects. With that said, the AWS S3 team really is impressive in what they've built out and deserves more shout-outs from people outside Amazon.

Could you expand more re: your minio experience? We're about to enter production with minio as our S3 interface-providing file storage system, and have found documentation to be sufficient.

However, I'm slightly concerned that we haven't tested it enough, and that doc and support may prove to be lacking (like in your case) when we hit edge cases and failure scenarios.

I'm checking it out too.

No LTS version. Bugs will be fixed in a new version, which may include new bugs. Some upgrades need full cluster restart. If you ask too many question you may get "it's better start a subscription".

We've been running for about a year now and had zero problems with it. (~20TB total, few single instances, serving as S3 backend to Restic - handmade HA - backup service)

What do you mean with "handmade HA" ?

Perhaps a high-availability system they made themselves?

That's the whole point of using it in the first place. And the solomon reed encoding/replication.

I highly recommend you look at leofs instead — it’s been absolutely rock solid for us.

It's a bit on a non-sequitur, but I am also looking at using MinIO as a S3 interface to a ZFS filesystem. Would be interested to hear from others about this use case for MinIO and possible alternatives.