Each machine that's spun up is built from scratch via one command-line call. The first half of the process interacts with each hosting API (we rely on DigitalOcean, Linode, and Vultr primarily), to build a clean slate machine with all of the packages and libraries that we expect.
The second half of the process runs the actual build process, building the instance step-by-step on top of the clean slate, blissfully unaware of which hosting provider it lives on.
This model allows us to be portable and avoid vendor-lock in, and a cross-provider infrastructure lets us gracefully handle system failures while keeping costs down.
My goal was the same, to make hosting more portable with features like snapshotting and restoration of WP sites across servers and to even eventually expand beyond just servers, to bring in domain registrars and cloud storage to be able to move things around easier. For example: you have a site hosted on AWS EC2 with DNS at Namecheap and nightly backups at Dropbox and let's say the AWS Virginia region goes down. You create a new server in Digital Ocean and restore the snapshot from Dropbox and the linked DNS at Namecheap is auto updated.
The more I thought about this though, I began to realize that maybe these features wouldn't be useful to the audience I wanted to target, which was people who wanted to grow from shared hosting and have something reliable and less noisy neighbors, but still more affordable than managed WP hosts and lastly more control (bring your cloud/server provider).
[1]: http://pagefog.com
But, building one is easier than it sounds! Think of the problem in two parts. First, find a distro used by multiple providers (we're on CentOS) and craft one script that uses each provider's API to spin up a clean machine.
Once you have that done, it's a matter of understanding your own build process, writing a script that you'll pass into each instance on creation that will fetch your source control, install libraries, and put all the pieces together.
Lots of if/else, lots of curl, lots of yum, lots of jq, but all of it is really straightforward.
Alternatively, pre-baking cloud images that have already run such a script and are ready to boot becomes pretty easy with a tool like Packer.
Though, as the underlying OS changes, you'll need something to validate your scripts' functionality against, and a tool that's a little more declarative might make them less fragile to those changes.
Having maintained various automation over the course of the past decade and a half, I can say things do change around. Over the course of only a few years though, obviously you can stick to some LTS release of whatever you're using and be pretty confident that e.g. "some-package" does not get renamed to "some-package-version" or split into "some-core" and "some-utils", or have a package get upgraded to a version with some less-than-backward-compatible configuration options, etc.
For us it's been about as transparent as a brick wall and I'm not clear if that's down to our bureaucracy or built into the design. Both are anathema to the goal of making complex deployments straightforward and self-describing (you can't manage something this complicated unless big parts of it are as obvious as can be).
TF can be tricky to grok at first especially if you don't have everything in TF. But, I couldn't imagine managing more than a server or 2 without it or something similar at this point. Once you get into VPCs, IAMs, etc..., some type of tool is really required.
I'm also a little confused about your transparency comment. IME, tf is very clear what it is going to do in a plan. The current state files are also just json, and easy to read/search if you're not sure about something.
As a general rule, if you're giving someone a tool that uses declarative syntax, you also need to provide them a private (not shared) sandbox in which to test out theories, try new things, and reproduce errors seen in production.
Since we don't have that, TF is pretty much the worst solution for our problems. Kube or even Docker Swarm would serve us much better.
https://github.com/sumdog/bee2
Vultr allows floating IPs for IPv4 and IPv6, but Digital Ocean only has floating IPv4. Vultr will start a machine with a floating v4, but you have to add a floating v6 address (giving you two v6 addresses). Digital Ocean does the same thing with v4 (giving you two v4 addresses). They both have different network adapter names, so you've got to configure those per provider as well.
Terraform and my own thing help in easing the transition if you ever need to move, but modifications will still have to be made.
Volume classes are probably the best example of being cloud-specific, but this problem is solved by having a different volume class for each cloud provider, named the same, such that the deployment can always grab a disk regardless of which cloud its living in.
There can of course be some difference in the capabilities that each cloud provider supports (e.g. not all load balancer implementations may support UDP) but the abstraction is definitely there.
https://github.com/kubernetes/enhancements/blob/master/keps/...
The idea would be to be portable, avoid vendor lock-in and take advantage price differences or quickly route around a system failure in one of the providers.