You say you don't want to use Puppet or Ansible but you are basically using kubernetes manifests for the same exact reason: configuration management. I know it can be funny and I totally support it but I thought it should be pointed out anyway.

The problem I have with traditional configuration management is that in the end, even if it's declerative, you are stil modifying a imperative OS/userland. So it will collect state at some point. Things like undoing changes with those tools is not that trivial. You have to actively reverse them in your configuration. Which turns nice CM code into mess. Want to try out something quick? Better not be afraid it messes up your OS/userland as there is no simple undo.

So since I'm doing isolation in containers/Docker already it's a small step to a lightweight Kubernetes. What Kubernetes gives me on top of that is that I can consider everything below the application layer as a declarative API.

not really true anymore with systemd portable services. or package managers.

k8s manifests are declarative though, not imperative config mgmt like the other tools.

Both Puppet and Ansible are declarative.

Why do you think people use them rather than shell scripts ?

That's maybe the theory, but in reality, the only thing Ansible hopefully is, is idempotent between playbook runs - but there are no guarantees there, at all. Only in very simple setups things can be fully declarative in it's totality.

Don't have much puppet experience, but I can't count the times anymore that I've had to add steps to playbooks just to determine stuff used in one of the following steps. The other option was to write a snowflake Ansible module. The individual steps/plays might be declarative, the playbooks are not.

They look declarative but every Ansible playbook I have ever read or written has involved some imperative code. And even if you only use it in a declarative fashion, it doesn't change the fact that it's very much a step-by-step ordered list of things to install.

The declarative syntax is certainly a step up from shell scripts, but it's not as pure as K8s.

Ansible is only declerative on a action level. At a playbook level it's imperative. You can install and remove the same package within a playbook, the outcome will be dependent on the order.

Puppet is fully declerative but for me it lacks an easy way to undo changes. It would be nice if it could work like Terraform where it keeps a 'state' of all changes it made in the past so when you remove a resource from your config it could 'undo' the change.

I still use Puppet (mostly with Bolt nowadays) for systems that don't fit Kubernetes, but they're becomming less and less.

This was indeed the motivation for my write-up :)

Thanks, I was meaning to look into this but your post will save me some research work.