I think the reason they didn't use a procedural language is because they specifically wanted to keep the playbooks declarative.

Ideally, the playbook should describe the state you want the system to be in.

This is 'almost' the case for most modules and I think this was a very good design decision.

Having said that, the ghastly YAML format was almost a deal-breaker for me too.

> ... keep playbooks declarative.

I guess I've never really considered playbooks declarative. They are written like and feel like a script I'd write in a language like, I don't know, Python. Now, Salt state files I would consider declarative. They are simply Jinja templated dictionaries (Ansible is YAML that tries to find values templated with Jinja; BIG difference as I've discovered using Ansible for the last 6 months) that actually describe a set of states the host should be in. Ansible's playbooks encode a set of function calls that should be applied in a particular order. Don't get me wrong, playbooks ARE powerful in their ability to describe processes that span multiple hosts and I recognize that, but I'd really like to use a real programming language rather than some half-baked one that is encoded in YAML.

I agree with the GP that I wish the Python API was a bit more robust. Currently we have several tools that generate playbooks on the fly, write them to temporary files and then run them with ansible-playbook to ensure we don't lose some functionality that makes playbooks as powerful as they are. Now, it appears the API has significantly changed in 2.0 and it looks like they expose (read: recommend using) more of it making it easier to script with Ansible.

Using a real programming language doesn't preclude a declarative programming style.

No, but using a 'real' programming language doesn't force you to write in a declarative style unless you use something like a pure functional language and before you know it people would start making a mess in their playbooks using the whole power of Python. Ansible is awesome for devops precisely because it restricts what you can do and forces you to try to do things in a consistent declarative (and hence idempotent) style, which is important to devops.

[edit: fix broken english]

This could have been solved by choosing a language like lua and restricting the runtime environment - which is what is done in game software. Lua also has a tight, efficient interpreter. Making a mini-programming language in YAML is _coding horror_.

> the playbook should describe the state you want the system to be in

This is the real reason.

The playbook is the final state. How it gets there is of no interest to you. All you need to be concerned with is the final result - the state described by the playbook. All the extra complexity is handled by the software.

But you can declaratively specify a full system configuration in a real programming language, too. This is precisely what the GNU Guix system configuration API does, for example. You write OS configs in functional, declarative Scheme code.

http://www.gnu.org/software/guix/manual/html_node/System-Con...

Ansible's playbooks aren't declarative for me, I've came to the conclusion that they're a framework to write shell scripts.

Dependencies are expressed by the sequence of the code blocks in the playbooks and roles, and I see them as implicit. In contrast, Puppet makes the dependency tree explicit with the "requires" language construct.

The thing is, Ansible works well enough for 85% of the cases, where the additional complexity that Puppet brings is not needed.

It's not the yaml part per se, which is easy to write, it's their weird mini-language they've encoded in yaml, with different vars coming from different places in different conditions.

Have to say that things are a lot better than when they weren't consistently using jinja2 to parse the strings.

Indeed I always liked Ansible, but the input format does occassionally get a little awkward. They've basically designed their own simple language semantically, but they constrained it to be expressed with the syntax of a data serialization language.

I think the choice of YAML is because Ansible is supposed to be declarative and a serialization format makes sense there, but some of the "metaprogramming" features they've layered on top like loops can feel a bit strange. Like I said I still like it, but I do feel occasionally that it might be nice to have another frontend that is more like a proper programming language.

Using python as the playbook API sounds dangerous. I think the idea is to restrict playbooks so they can be sanitized.

To me, it still seems like a win compared to over-engineered solutions like chef.

I see absolutely no point in this. Code is data, and data is code. Why are Ansible users so eager to cripple themselves?

Do you know that some people use jinja2 templates to generate their YAML playbooks? Do you know what could have helped them? A real programming language.

This was posted on hacker news sometime ago,

https://news.ycombinator.com/item?id=10567408 -> http://lukeplant.me.uk/blog/posts/less-powerful-languages/

I think it is relevant to this discussion. Having used Puppet before ansible, I tend to agree with it.

The point is that it's not turing complete. Thus it's more readable, less susceptible to technical debt and less susceptible to bugs and always by definition, declarative.

Turing complete code turns into an unreadable mess far more easily.

But isn't in fact Turing complete?

It has variables, control structures, loops and exceptions.

That does not make it Turing complete.

"You need some form of dynamic allocation construct (malloc ornew or cons will do) and either recursive functions or some other way of writing an infinite loop. If you have those and can do anything at all interesting, you're almost certainly Turing-complete." -- http://stackoverflow.com/a/449170

There is a do-until loop: https://docs.ansible.com/ansible/playbooks_loops.html#id22

Which satisfies the infinite loop issue.

Have not looked carefully at the dynamic allocation, but I am guessing (hand waving here....) that it could be accomplished.

Power comes at the price of complexity. If you want to write your playbooks in lisp, ansible is not the way to go.

I see your viewpoint, but I personally feel that YAML is a much better solution. I don't want to run arbitrary code in my playbooks! I just want to be able to trigger ansible modules.

Besides, it's really easy to write your own Ansible module (in Python), which seems to solve the issue for any use case I can think of?

You've never wanted to do something as simple as map over a list in your configuration to handle repetition?

You're talking about eliminating duplicates in a list right?

    my_list | unique

I think this is what you're looking for. Any transformation of data that can't be done with the standard filters can be done with a simple filter plugin written in python.

>You're talking about eliminating duplicates in a list right?

No, I'm talking about any atbitrary list processing operation. Some combination of map, fold, and filter, for example.

So, in order to do that in an Ansible playbook, I need to write a Python function to do it and use it in the YAML file as a filter because the DSL isn't expressive enough to do it on its own. So... why wouldn't I want to just use Python again?

I feel as though I do process lists via ansible. You could be less abstract and provide code. Otherwise, it is hard to take this complaint seriously.

Here we install Jenkins jobs....

  - name: create the job config.xml files
    sudo: true
    template:
      src={{ item }}
      dest={{ conf_jenkins_home }}/jobs/{{ item | basename | replace(".xml.j2", "") }}/config.xml
      owner=jenkins
      group=jenkins
      mode=0600
    with_fileglob:
      - ../templates/jobs/*
    notify: restart Jenkins

And install SonarQube plugins...

  - name: install sonar plugins
    sudo: true
    get_url:
      url={{ item.value.url }}
      dest=/opt/sonarqube/extensions/plugins
      owner={{ ansible_ssh_user }}
      group={{ ansible_ssh_user }}
      mode=0755
    with_dict: "{{ sonarqube_plugins }}"

You mighh be interested in looking at fabric in that case - similar idea of running commands over SSH via python. The downside is that it isn't as fully-featured because you have to write the primitives yourself, but it does allow you to write simple recipes with ease.

Have you looked into SaltStack at all? The default is to write states in YAML+Jinja but you can change it to use pure python, or a PyObjects renderer that gives it a very pythonic API.

in what way is Chef over-engineered?

Allow me to refer you to my favorite bit of Chef documentation https://docs.chef.io/attributes.html#attribute-precedence

Also the Chef server is more complex and written in more languages than the stack of an entire early-stage startup https://docs.chef.io/chef_server.html

And the server uses its own versioning, because what I really need is something competing with my DVCS

Too many things to count, really

To make this a fair comparison, you should be comparing chef-solo (or chef-zero) deployments to Ansible since the closet thing Ansible has to a dynamic source of truth is dynamic inventories. But we'll table the discussion of whether or not I should be required to build my own CMDB from scratch or not for now...

The Chef Server stack is somewhat complicated, though with an omnibus install you will rarely have to interact with it at that level. These days, it scales well up to thousands of client nodes with no meaningful effort.

I'm not sure what you mean by the server using its own versioning. Cookbooks are versioned, as they are deployable artifacts. Environments then define which versions of all the cookbooks exist, so you have an easy promotion strategy. The challenge with Chef Server and DVCS is that you have to centralize the uploading of Chef artifacts (roles, environments, data bags, cookbooks) to Chef Server if you have a team with more than 1 person. That challenge is not unique to Chef, however. Hopefully, a team of Ansible users are not running Ansible playbooks directly from their workstations. If you try to upload a version of a cookbook that is older than the current version, it yells at you, so it makes certain bad behaviors more difficult.

There is plenty I don't like about Chef. And Puppet. And Ansible...I've worked with all of them, but I still haven't seen a valid argument to support the phrase "over-engineered" when used on Chef. Of all the configuration management products out there, Chef is still the most flexible and the most powerful.

>The Chef Server stack is somewhat complicated, though with an omnibus install you will rarely have to interact with it at that level.

Omnibus is the worst thing to happen to packaging. It bundles its own copy of everything.

Is anyone that's not an Opscode employee capable of building Chef Server from source? I tried and failed. Software that can't reasonably be built from source is proprietary in practice. I want to run Chef Server on Debian but I can't because its seemingly impossible to build, so I can only use a platform that Opscode provides a pre-built binary for. It's a terrible situation to be in.

Here's one select example: https://docs.chef.io/attributes.html#attribute-precedence

There's plenty of this stuff all over.

In what way is attribute precedence over-engineered? You can ignore most of it for most cases and just use default for everything, but when you need it, it's there for you. I don't see how it's more or less simple than what Ansible has defined: http://docs.ansible.com/ansible/playbooks_variables.html#var.... I count 16 precedence rules in Ansible 2.0.

Every time a new configuration management utility comes out, everyone loves it because of it's simplicity or some arbitrary measure of "lightweight," but once Systems Engineers need to solve real automation problems in the real world, they start adopting features to work around the illusion of being "lightweight."

I like Ansible for automating my home network where I have simple problems and no need for environment separation. For professional work, I stick with Chef or Puppet.

It's got a lot of complex rules to memorize, which aren't exactly intuitive either.

I don't think that's a very good example. Attribute precedence is easy to understand, and it's very necessary. After working with it for a bit it becomes very intuitive.

And all of the config management tools have a comparable concept. Ansible 2.0 has 16 levels of variable precedence, Puppet has hiera which gives you arbitrary levels of precedence, and Chef has attributes, of which there are 11-15, depending on how you count.