News for hackers. Stuff that matters.
Re:Show HN: Interactive Ansible Tutorial by jasim on Friday January 26, 2018 @04:37AM (#16237965)
I wish there was a server provisioning "library" written in an ML-ish language.

The trouble with even a simple, well-made tool like Ansible is its dynamic DSL. The YAML file is essentially code, but modules like "file" and "apt" written as configuration entries give a declarative feel to it. But they are just procedures that strive to be idempotent, and its YAML file is just a series of procedure calls, sprinkled with some facility for organizing hierarchies and sharing code between them.

YAML the format is also not a great way to encode systems data. Thankfully Ansible has documented the gotchas with its space-indentation and need for explicit quoting in http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch.... But that is just an attempt to band-aid over the unsuitability of YAML for this purpose.

Chef's configuration on the other hand can be done in Ruby, but you're still constrained to its DSLs. The shape and structure of Chef's objects can only gleaned by reading reams of documentation, and worse, by running the cookbooks with debug statements.

The problem is compounded when you upgrade these tools and things stop working silently. The community solution for this unsurprisingly is unit tests. I'm still grappling with that idea.

But what if we could write our own provisioning code in Haskell or OCaml/Reason, with library functions that help install packages, returns errors in neat sum types which we can confidently handle and doesn't leave us to worry about the known unknowns that we haven't enumerated and handled, and a compiler with exhaustive pattern matching that helpfully tells us about all the record types and their structure within a fast edit-compile cycle, ...

Re:Show HN: Interactive Ansible Tutorial by arianvanp on Friday January 26, 2018 @05:31AM (#16238182)
This exists! and it's pretty awesome to use! https://hackage.haskell.org/package/propellor
Re:Show HN: Interactive Ansible Tutorial by jasim on Friday January 26, 2018 @05:39AM (#16238209)
You wonderful person!

"Properties are defined using Haskell in the file ~/.propellor/config.hs. There is fairly complete API documentation, which includes many built-in Properties for dealing with Apt and Apache , Cron and Commands , Dns and Docker, etc.

There is no special language as used in puppet, chef, ansible, etc.. just the full power of Haskell. Hopefully that power can be put to good use in making declarative properties that are powerful, nicely idempotent, and easy to adapt to a system's special needs!"

I'll be learning Haskell for real this time.

Re:Show HN: Interactive Ansible Tutorial by breatheoften on Friday January 26, 2018 @12:59PM (#16241279)
As a former ansible user -- I'm completely there. There are lot of things to like about ansible but the unwieldy ad-hoc non-composable task description language and the lack of static verification for anything makes the whole experience so much worse than what I think is possible in this space. Ansible has a 'fact gathering' stage which is the first thing that happens after connecting to a host -- I've been dreaming about a system that is capable of leveraging an execution model like ansible's for 'gathering facts' from a set of hosts -- and then turning those facts into statically analyzable types in the context of some (statically typed) language -- with convenient language-level/library-level interfaces for expressing common orchestration/system-admin/stateful-scripting logic. I think such a system would be _EPIC_ to use.
Re:Show HN: Interactive Ansible Tutorial by emmelaich on Friday January 26, 2018 @09:00PM (#16244410)
Facebook's osquery could help here.

You just(1) have to implement update functions as well.

(1) that's the "just" that doesn't mean "just" :-)

Re:Show HN: Interactive Ansible Tutorial by orev on Sunday January 28, 2018 @04:17PM (#16253559)
What have you moved to if no longer using Ansible?
Re:Show HN: Interactive Ansible Tutorial by susi22 on Friday January 26, 2018 @07:07AM (#16238525)
I'm a Clojure dev so I really like fast edit-compile cycles. Note, Ansible will happily take in JSON, so you can take your favorite language and write all the playbooks in that language. Just generate the playbooks on the fly (ie. genearte JSON not yaml) and shell out to ansible to run it. So you have all the features of ansible available but can arbitrarily build up your playbooks in your favorite language.

It's hacky code, but that's exactly what I did in Clojure: https://gist.github.com/rauhs/a72cfbeef4c80f9a58480484c49e7a...

With a normal REPL this works well especially when developing plays.

Re:Show HN: Interactive Ansible Tutorial by emmelaich on Friday January 26, 2018 @09:02PM (#16244416)
> Ansible will happily take in JSON

Yes, JSON is very close to a proper subset of Yaml.

Re:Show HN: Interactive Ansible Tutorial by vesak on Friday January 26, 2018 @08:31AM (#16238947)
Not exactly the same, but http://nixos.org would give you some of what you want.
Re:Show HN: Interactive Ansible Tutorial by emmelaich on Friday January 26, 2018 @05:01AM (#16238057)
Yeah, me too (a stronger / more statically) typed language.

But you're up against a potentially changing state -- the file system. So that only gets you so far.

Idempotency is very hard to achieve. What gets me is there is an excellent tool -- rsync -- which is extremely fast and idempotent. The various config tools make far too little use of it.

Re:Show HN: Interactive Ansible Tutorial by gshakir on Friday January 26, 2018 @08:57AM (#16239133)
I don't wish that. How many DevOps people you know that can grok Haskell? There is a reason why Ansible is so popular, the barrier to entry is so low and that's the way it should be.
Re:Show HN: Interactive Ansible Tutorial by drcongo on Friday January 26, 2018 @08:53AM (#16239094)
I came to Ansible from Puppet and Chef because in the early days, it was way easier to get going. But with almost every release it gets worse, and so much of this is down to the YML / Jinja hybrid DSL. We have a fantastically powerful fabfile (Python/Fabric) that we include in every project, it keeps getting more powerful over time because anything we need it to do we can just write the code to do it. Our Ansible roles are a real bind to maintain though. I'll definitely take a look at Propellor recommended below, thanks.
Re:Show HN: Interactive Ansible Tutorial by xref on Saturday January 27, 2018 @02:22AM (#16245225)
hasn't fabric been abandonware on python 2 for years?
Re:Show HN: Interactive Ansible Tutorial by drcongo on Thursday February 01, 2018 @05:30AM (#16280793)
Sort of, there's a Python 3 fork of it though which is solid.
Re:Show HN: Interactive Ansible Tutorial by gtirloni on Friday January 26, 2018 @07:47AM (#16238711)
This isn't a replacement for configuration management but allows you to directly program into code how the application will be deployed. It seems interesting.

https://metaparticle.io/about

Re:Show HN: Interactive Ansible Tutorial by crdoconnor on Friday January 26, 2018 @04:48AM (#16238005)
>The YAML file is essentially code

YAML is not turing complete and is hence not "code". That is a large part of what makes ansible such a powerful tool - it enforces a strict separation of concerns between the definition of server configuration and its execution and keeps one in a language that is deliberately not able to be made more complex than is needed, is thus kept readable and straightforward to maintain.

By contrast, turing complete languages are veritable magnets for technical debt, which is why a bad chef playbook will always be worse than a bad ansible playbook. Haskell, being turing complete, would suffer the same issue albeit not to the same degree as ruby.

>YAML the format is also not a great way to encode systems data. Thankfully Ansible has documented the gotchas with its space-indentation and need for explicit quoting in http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch.... But that is just an attempt to band-aid over the unsuitability of YAML for this purpose.

YAML has a few oblique gotchas but they don't affect the fundamental suitability of it for this purpose. Once you get used to it, space indentation becomes a feature, not a bug.

Re:Show HN: Interactive Ansible Tutorial by jasim on Friday January 26, 2018 @05:02AM (#16238066)
Any sufficiently complicated configuration language contains an ad hoc, informally-specified, bug-ridden, slow implementation of a Turing complete programming language.

Ansible playbooks have variables, loops, conditionals, scoping rules, inheritance, and you can even extend it by writing custom modules in Python, which is a Turing complete language.

The problem with not using a real programming language when you're trying to essentially "program" is that you're expressing complex programs without any of the abstractions and tooling support that a real language has.

Despite all that I like Ansible's constrained format and the resulting simplicity compared to Chef. But I think we can take those design aesthetics and apply it in a statically typed library and maybe get something more sturdy.

Re:Show HN: Interactive Ansible Tutorial by jmcnulty on Friday January 26, 2018 @09:24AM (#16239332)
> and you can even extend it by writing custom modules in Python

Actually you can write custom modules in any language proving when you exit you provide the required JSON to indicate if the module succeeded or failed, changed something, or has a message to pass back.

Re:Show HN: Interactive Ansible Tutorial by crdoconnor on Friday January 26, 2018 @05:17AM (#16238126)
>Ansible playbooks have variables, loops, conditionals, scoping rules, inheritance

As far as I am aware, variables, scoping rules and inheritance don't imply turing completeness and the way it does loops and conditionals could really more be characterized as configuration for loops/conditionals further down the stack (e.g. as seen here http://docs.ansible.com/ansible/latest/playbooks_loops.html#... )

>you can even extend it by writing custom modules in Python, which is a Turing complete language.

Yes, the idea is that you have the power to break into a (good) turing complete language when it is necessary but you can keep that code - and by extension the technical debt it would attract - to an absolute minimum. I've personally never felt the need to write a custom module in python when using ansible despite using it a lot, so for me, minimum is zero. I'm sure I'm not alone either.

>The problem with not using a real programming language when you're trying to essentially "program" is that you're expressing complex programs without any of the abstractions and tooling support that a real language has.

Yes, this is a problem if you're trying to work within a problem space that is not heavily constrained. If almost every time I used ansible I felt the need to write a python module to do almost everything I would agree that it is a fundamentally unsuitable task. I feel this issue when writing build scripts which I think absolutely have to be kept turing complete, but never server orchestration.

Re:Show HN: Interactive Ansible Tutorial by leg100 on Friday January 26, 2018 @05:52AM (#16238257)
> As far as I am aware, variables, scoping rules and inheritance don't imply turing completeness and the way it does loops and conditionals could really more be characterized as configuration for loops/conditionals further down the stack (e.g. as seen here http://docs.ansible.com/ansible/latest/playbooks_loops.html#.... )

Then what does imply turing completeness?

According to the wikipedia article [1]:

"To show that something is Turing complete, it is enough to show that it can be used to simulate some Turing complete system. For example, an imperative language is Turing complete if it has conditional branching (e.g., "if" and "goto" statements, or a "branch if zero" instruction; see one instruction set computer) and the ability to change an arbitrary amount of memory (e.g., the ability to maintain an arbitrary number of variables). Since this is almost always the case, most if not all imperative languages are Turing complete if the limitations of finite memory are ignored."

[1] https://en.wikipedia.org/wiki/Turing_completeness

Re:Show HN: Interactive Ansible Tutorial by crdoconnor on Friday January 26, 2018 @07:09AM (#16238532)
>Then what does imply turing completeness?

If it did arbitrary branching and if you could mutate a variable and then reuse it later on in a playbook.

Re:Show HN: Interactive Ansible Tutorial by Spivak on Friday January 26, 2018 @09:31AM (#16239388)
The branching is a given. Variable mutation is totally possible with set_fact.
Re:Show HN: Interactive Ansible Tutorial by emmelaich on Friday January 26, 2018 @04:53AM (#16238021)
I agree with most of your comment but

> YAML is not turing complete and is hence not "code".

seems wrong to me. In fact I've heard code/coding to include non-Turing complete languages (e.g. html) as distinct from programs/programming.

Re:Show HN: Interactive Ansible Tutorial by guywhocodes on Friday January 26, 2018 @05:00AM (#16238053)
I think SQL is another prime example, being able to solve a problem effectively without resorting to Turing-completeness is preferable to me, as long as that language is simple enough.
Re:Show HN: Interactive Ansible Tutorial by gshakir on Friday January 26, 2018 @10:34AM (#16239896)
Yes agreed, Declarative languages like SQL or Ansible are great for most of the use cases. You can always use extensions to get down to imperative steps.
Re:Show HN: Interactive Ansible Tutorial by DerBesserWisser on Friday January 26, 2018 @05:24AM (#16238157)
HTML is a markup language and not a programming language.

IMHO you can only "code" in programming languages (not necesseraly executed by a CPU).

A programming language does not need to be turing complete though to "code" in it. You're just limited to the problems you can solve with a turing-incomplete language.

It's interesting to think if using a problem-declaration language is coding.

Re:Show HN: Interactive Ansible Tutorial by dozzie on Friday January 26, 2018 @07:45AM (#16238704)
>> The YAML file is essentially code

> YAML is not turing complete and is hence not "code".

You do realize that YAML is a data serialization format, and as such is not a programming language by any stretch, right? And it's the Ansible's language encoded as YAML that is the problem? And I wouldn't be so sure that the language is not Turing complete, especially that this YAML is templated with Jinja2 (or has something changed recently?), which is Turing complete.

Re:Show HN: Interactive Ansible Tutorial by Aissen on Friday January 26, 2018 @05:19AM (#16238135)
If only it were that simple. There are enough filters, that this is not true IMHO. I haven't gotten around to prove it yet, but I think it could be proven that the playbook language is turing-complete. Others have the same hunch:

https://stackoverflow.com/questions/40127586/is-ansible-turi...

Re:Show HN: Interactive Ansible Tutorial by lima on Friday January 26, 2018 @11:22AM (#16240349)
I fully agree.

While technically inferior to Chef and Puppet, the simplicity of Ansible's syntax is what makes it so useful.

Learning curve and preventing people from being "clever" is absolutely essential for a tool which is used by a team of developers.

Re:Show HN: Interactive Ansible Tutorial by zeckalpha on Friday January 26, 2018 @07:46AM (#16238707)
This is why I wrote YAPL: https://github.com/kmarekspartz/yapl/
Re:Show HN: Interactive Ansible Tutorial by emmelaich on Friday January 26, 2018 @09:05PM (#16244423)
Brilliant. You must be reading my mind.

> Inspired by YAML, Scheme, and TCL. Based on ulithp. Converts YAML to s-expressions then evaluates using ulithp.

Re:Show HN: Interactive Ansible Tutorial by hedwall on Saturday January 27, 2018 @07:36AM (#16245980)
But ansible playbooks are not just YAML. They’re YAML with Jinja2.