At some point in a previous company we had a lot of individual VPSes set up basically the same way. I was sick of internal documentation that listed step-by-step commands intertwined with descriptions and manual actions, and any attempt at puppet or ansible just blew up because it was something else to learn by the team (believe me, I tried, it just wouldn't stick with anyone).

So I created `apply`[0].

This small bash thing pushes bash scripts called 'units' through ssh to execute through an uploaded 'run' script:

   ./push units/update units/sshd units/ssh_authorized_keys [email protected]

By writing those 'unit' scripts to be idempotent you can just run them again and again. 'units' can be aggregated in 'groups', which can themselves reference 'groups':

   ./push groups/base groups/ruby units/dockerd [email protected]

Finally, you can define 'hosts', which are like 'groups', only they save you some typing, which it could do in sequence or parallel:

   ./apply hosts/foo.example.com hosts/bar.example.com

And since units/, groups/, and hosts/, are just directories and files, autocompletion works immediately and you could get creative with shell expansion for arguments.

It was a deceptively simple experience, immediately accessible, trivially enabled literate coding, and overall extremely useful both to set up and maintain those VPSes as well as creating dev environments, or local VMs to test a e.g one-shot unit performing a change or migration.

[0]: https://gitlab.com/adhoc-gti/apply

[1]: https://github.com/lloeki/apply (personal fork)