Pretty sure you're not doing anything wrong - well, if you are then I am too :-)

What's often overlooked (I believe) is that when you're doing this work in your day job you've for existing infra to support your infra along with other folks you can rely on to for help.

With home infra, you first must invent the universe (apologies to Carl). Having built 3 variations of home infra (Swarm on x64/Beelink, K3s on Pi, K3s on x64/Odroid) I've gained a strong admiration for anyone who take this on irregardless of there relative success.

What I've learnt over time is to add as little accidental complexity as is possible which I think is what you're getting at. One incarnation of the Pi K3s was provisioned by Ansible (on it's own stand alone Pi that the cluster would netboot from). Was Ansible better that imaging the usb drives manually and running a small script on each of the nodes? - probably a wash. I did however learn a bit of Ansible.

Today's stack is way too complex, and fragile. The entire stack relies on the goodwill of other projects to continue to be maintained/not have breaking changes. Even with the initial setup cost s(h)aved there will be a continuous maintenance burden. What you get in abstraction and automation you pay back in babysitting the additional connecting interfaces.

> The entire stack relies on the goodwill of other projects to continue to be maintained/not have breaking changes

I agree with the sentiment of your comment, but when was this ever not the case, other than the days when you built your own OS and tooling from scratch?

It’s a good habit to always use tools like terraform for cloud or Ansible/Salt/Puppet for machines instead of directly doing something.

Especially cloud setups that just run containers are relatively easy to get idempotent with terraform

No. It never gets a habit. It is torture, and you have to look up the commands every time you do it, because you do it infrequently enough not to learn by heart.

Huh? Compared to... running commands infrequently to administer servers? IaC is strictly and comically better.

Commands you use every day in the CLI? In your editor?

I can trivially enter Emacs and modify Apache config when it breaks backwards compat between distribution upgrades, but when it's infrastructure as code...

I can’t think of a single time I’ve had an Apache config break with upgrades - other than having to recompile the webservice proxy module to change the case of the Upgrade header for a device that doesn’t work with the standards and is case sensitive

(Major upgrades every couple of years break the binary compatibility)

I am pretty confident it broke for me between 12.04 and 14.04 Ubuntu releases, and I am pretty sure both were already in the 2.x series.

Most likely the change in NameVirtualHost behaviour as listed at https://httpd.apache.org/docs/2.4/upgrading.html, but there are a bunch of other changes too.

Oh sure every few years when you upgrade to a new OS, but not a normal security upgrade.

Still need to install the machine with the "hand" or with Satellite or Anaconda.

(Repo owner here) You're not wrong; it's still not easy for me even though I do this in my day job.

Projects are making this easier. For example moving from large mono config files to using conf.d/ directories where you can drop in extra files and manage each one independently is great for IaC.

While it's challenging up front I do enjoy being able to freshly install the latest version of Fedora, run my playbook and be more or less up and running.

It feels cleaner and more reliable (at least until this week when a broken selinux policy made it to stable) rather then trying to upgrade packages across major release versions in place.

<hat material="tinfoil"> If I've somehow acquired secret hidden malware or someone has opened up something to come back in later that's also going to get flushed out at least once every six months.