I've never been a fan of chef, vagrant, puppet, ansible and tools like that. They don't relieve you from the responsibility of knowing how to sysadmin and knowing bash so I don't find them worth it.

Ideally my "runbook" has text instructions to download and run a script, otherwise it has more steps.

The hierarchy of tools I'd use for automation is:

  * bash
  * python
  * java

An unusually complex one I built constructs and tears down systems in the cloud that do some work, such as booting up a AWS instance that contains a database server, building a large database, then making an AMI. The Java program sets up the instance and the resources the instance needs and writes a bash script that the instance runs when it boots up. When the instance is done doing its thing, it posts a message to the Java program that shuts the instance down, makes the AMI, etc.