1 https://github.com/prometheus-operator/prometheus-operator/b...
I can see you don’t like operators and that’s fine. Many folks find it’s a useful pattern to follow
Operators are just the non-containerized daemons of the Kubernetes OS. We did all this work to run everything in neatly encapsulated containers, and then everyone wants to run stuff globally on the whole cluster. What's the point? Do we just containerize clusters and start over?
Migrations can be run in containers (and they are, even with the operator), but it's actually a lot of work to run them at the right time, only once, with the right flags, in the right order, waiting for SpiceDB to reach a specific spot in phased migrations, etc.
Moving from v1.13.0 to v1.14.0 of SpiceDB requires a multi-phase migration to avoid downtime[0], as could any phased migration for any stateful workload. The operator will walk you through them correctly, without intervention. Users who aren't running on Kubernetes or aren't using the operator often have problems running these steps correctly.
The value is in this automation, but also in the API interface itself. RDS is just some automation and an API on top of EC2, and I think RDS has value over running postgres on EC2 myself directly.
As for helm charts, this is just my opinion, but I don't think they're a good way to distribute software to end users. The interface for a helm chart becomes polluted over time in the same way that most operator APIs become polluted over time, as more and more configuration is pulled up to the top. I think helm is better suited to managing configuration you write yourself to deploy on your own clusters (I realize I'm in the minority here).
[0]: https://github.com/authzed/spicedb/releases/tag/v1.14.0
You run the operator itself in a pod, but you can only have one of them running at a time, hence they are global. You can run as many Alpine/Ubuntu/... containers as you want, and as many ElasticSearch or PostgreSQL as you want, in isolation, that's the beauty of containers. But you can only run one SpiceDB operator or one ElasticSearch operator, on the whole cluster. We're back to square one.
That said, I do think the direction that Kube has gone with CRDs, by leaning into making them more like built-in apis instead of allowing them to be differentiated as an external extension point has limited what you can do with them.
When ThirdPartyResrouces just carved out a group and a name in the API it was simple to have multiple copies and multiple versions running together in a cluster. But that has become more difficult with time, especially with ConversionWebhooks - a feature that forces all controllers for an API to agree not just on a version, but on the specific ways versions convert into each other.
In the latest release of the spicedb-operator[0], I added a feature that allows users to specify arbitrary patches over operator-managed resources directly in the API (examples in the link).
There are some other projects like Kyverno and Gatekeeper that try to do this generically with mutating webhooks, but embedding a `patches` API into the operator itself gives the operator a chance to ensure the changes are within some reasonable guardrails.
[0]: https://github.com/authzed/spicedb-operator/releases/tag/v1....