Re:Ansible 101 by Jeff Geerling – new series on YouTube
by geerlingguy on Sunday March 22, 2020 @02:00PM
(#22656984)
In addition to announcing this new streaming series (hope it's helpful if you don't know Ansible yet!), I wanted to also remind anyone who hasn't taken me up on the offer yet: Ansible for DevOps[1] and Ansible for Kubernetes[2] are available free on LeanPub the rest of this month!
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by YPCrumble on Sunday March 22, 2020 @05:37PM
(#22658931)
Thank you for your incredible contributions to Ansible. I help maintain an Ansible script for Django applications (https://github.com/jcalazan/ansible-django-stack) and your blog and repos have been invaluable. Your blog posts on testing using Molecule, your htpasswd role, and so many others have come in handy.
I'll add these links and your YouTube channel to our README now :).
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by boomskats on Sunday March 22, 2020 @05:32PM
(#22658884)
Hey Jeff, this is awesome. Ansible for DevOps was my bible for a long time - I wasn’t aware the k8s book was even in the works! I’ll definitely be taking you up on that.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by BiteCode_dev on Sunday March 22, 2020 @05:09PM
(#22658676)
I'm hopping someday people will provide a friendly Python API on top of ansible so that I don't have to write its YAML DSL anymore.
The "it must be idempotent" argument lead to putting conditional into a a language that was never meant to be, doing the whole "store" danse that could be done with simple function calls and whole lot of conventions instead of a proof tested import system.
It's calling python code behind the scene anyway, and nothing reads the playbook apart from ansible itself.
Ansible is a great tool, but I strongly believe the experience would be 100% better without learning, crafting and debugging those twisted .yml.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by justinsaccount on Sunday March 22, 2020 @05:29PM
(#22658858)
> It's calling python code behind the scene anyway
It doesn't even really do that. It sets up a rube goldberg mechanism to push a python program containing a base64 encoded zip file containing the module it wants to execute it. Then it runs a shell to run python to run that module. This is why ansible is 2000x slower than it should be.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by geerlingguy on Sunday March 22, 2020 @07:04PM
(#22659606)
Mitogen is a nice speedup if it works for your use case. Some of the improvements from the mitogen architecture have slowly worked their way into Ansible core, but it seems that a number of the major shifts can lead to breakage in some scenarios. Hopefully more of the low hanging fruit can make it into Ansible someday...
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by throw0101a on Monday March 23, 2020 @11:15AM
(#22664875)
I have no idea. AFAICT these were developed because some people found Ansible too limiting, so instead of the 'narrowness' of Ansible modules you can program just about any behaviour you want.
However, I've said this many times before—if your playbooks start mixing Python into tasks, you're either doing something wrong (it could be architected better), or it's time to write a custom Ansible module so the Python goes into a `.py` file and not into YAML.
99% of my playbooks are strict YAML with maybe a few Jinja filters, which make everything very readable. But I've also seen a ton of YAML that makes me want to barf.
from icandream import playbook
pb = playbook("Hello Ansible", hosts=["ansibleclient01.local"])
with db.define_task(Copy '/tmp/testfile.txt' into 'hello world') as task:
task.copy(content="hello worldn", dest="/tmp/testfile.txt)
Instant benefits:
- Python interpretter is very good at telling you error messages.
- You get an exception system, stack trace and a debugger for free
- The entire python tooling ecosystem, including full monthy IDE support, linters and formaters, is at your disposal
- Any more complex example is going to be much cleaner than the yaml version, exponentially. And no need to live in the "yaml/python module" dichotomy to keep PB clean.
- No need to reinvent the wheel. Python has elegant import, namespaces, ways to pass data around, conditionals, etc.,
- You can import Python libs. Event if it's just for declarative stuff like pendulum, appdirs..
The only cons are:
- idempotence needs to be enforced culturally. It's already the case for people writing the module behind the scene, and a good API + some warning tricks would be enough IMO.
- can't read the files outside of Python. But nobody does it anyway, and we can always let the Python files output the yaml representation if needed.
Now I don't hold my breath because it's a lot of work and the ansible community usually don't like the idea at all, and I'm not willing to do it. Fair is fair.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by mindwok on Sunday March 22, 2020 @05:22PM
(#22658790)
Yes! Totally agree. Ansible's largest problem for me is trying to wrangle YAML into a domain it doesn't belong. This is why I prefer puppets approach, which is a domain specific language that maintains the idea of YAML in that its mostly declarative, desired state, but it still provides sane constructs for looping, conditionals and types. Ansible would benefit greatly from this.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by geerlingguy on Monday March 23, 2020 @12:14AM
(#22661208)
The main problem I have with any kind of DSL (Python, Ruby, PHP, Go-based, whatever) is that it usually doesn't abstract enough of the underlying language away, meaning if you're not already a [language XYZ] developer or willing to learn some of the ins-and-outs of the underlying language, you're not going to be very productive.
With Ansible, you can go _very_ far before you have to know anything about Python (well, most of the time—annoyingly, there are situations like naming groups where Python's variable syntax leaks through currently).
It would be like making people know basic Go syntax just to write or edit a Kubernetes manifest—that immediately limits its appeal to those who have at least a passing interest in Go (which is a much smaller audience than 'willing to learn YAML, which is used all over the place').
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by _ix on Sunday March 22, 2020 @05:37PM
(#22658928)
I’ve worked with Puppet, Ansible, and Salt professionally in the past few years. Of the three, I’m unsure which I’d choose for greenfield, but I think I would rather inherit a legacy Salt codebase than one in Puppet or Ansible.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by blakesterz on Sunday March 22, 2020 @05:46PM
(#22658997)
I've been working with puppet and ansible, and I'm sure I'd rather take on an ansible. I've been exposed to both and ansible has been far easier to figure out for me. Obviously this will be different for everyone depending on what you already know and who wrote that thing that you're inheriting.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by notabee on Sunday March 22, 2020 @07:18PM
(#22659719)
Ansible seems to be better at declaring a particular configuration at build time, which should be repeatable and tested, and then perhaps running limited updates in a more controlled fashion later. The destination hosts should be disposable and recreated for any significant changes. If you write playbooks to change existing state frequently you have to be very careful to write in tasks that check the previous state. If you can version the roles and playbooks carefully then you can have some idea of the systems' state. For simple idempotency, like changing an app's config template and restarting a daemon, the modules have you covered, but for more complex dependencies between services it can get hairy. Some unpredictable things can happen. Example story: systemd-tmpfiles on CentOS 7 has a poorly documented behavior of skipping files in /tmp or /var/tmp that are both root-owned and read-only. It's not something in the options or config, just a default behavior (that was removed in later versions). We had a playbook that would clone a git repo into /var/tmp and then copy the required files into their appropriate place. The git module worked fine to update the repo each run, until several weeks passed and systemd-tmpfiles blew away most, but not all, of the repo files. Some files with just read permission remained. The module could handle either all the repo files existing or having the repo disappear just fine, but not a partially deleted repo. It would error out on that task. So now there's a new dependency on a changed systemd-tmpfiles config to skip that directory, and the partially deleted mess must be deleted for a run to work on existing hosts with the old config. There's lots of stuff like that that can accrue.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by expialidocious on Sunday March 22, 2020 @06:23PM
(#22659309)
I've seen molecule mentioned a lot, but I've never used it. My understanding is that it will spin up a container, vm or cloud image for you, run your playbooks, validate the result and tear down for you. Seems like a lot of trouble. Red Hat maintains it, I believe.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by joshvm on Monday March 23, 2020 @02:03AM
(#22661611)
I use a combination of Ansible and Docker. I use it for deploying to embedded platforms where sometimes you just want to reflash the board, or you have to upload a new kernel image for some reason. Or of course when a new board turns up! The goal is so that a new user can install everything from scratch with one command (which could be wrapped into a pretty GUI). I didn't want to go with storing a flash image because it's a lot of space and not portable. This setup will work on basically any dev board with minimal changes (the docker image is platform agnostic and can also be tested on amd64 as well as arm7 and aarch64). It took a while to set up, but I like using an approach which is very ephemeral - means you're forced to document exactly what you did to get the system to be in that state. And you don't worry that if you fry a board that it'll take days to set up again.
Ansible makes it easy to do mundane things like delete the default user and set up a new account, copy over ssh keys for easy access, add services and set up the docker container. Docker has the meaty stuff that takes ages to build and install, so I make sure it works and then save the image for easy download later.
What I've currently got going is everything in one repository, it has the playbooks, docker files, tests, service files, etc. I clone that using ansible on the target machine. Ansible then pulls the repo to get new changes and rebuilds the container. The container also adds that repo as a volume so that tests and other things are accessible. I don't store containers, I just spin up a new vm on boot.
I do have a "backup" playbook which installs everything inside the container when I want to test outside docker. This has happened a few times when using custom libraries that are specific to the embedded platform that don't come with the docker images (eg hardware accelerated gstreamer plugins come to mind). On the Pi life is a bit better because there are raspbian docker images anyway.
Re:Ansible 101 by Jeff Geerling – new series on YouTube
by geerlingguy on Sunday March 22, 2020 @07:06PM
(#22659617)
Most of the time I'm using Ansible to build containers or AMIs (or other images), which are then deployed, and overwritten for updates.
Other times, I write a secondary playbook to roll back some deployment if needed. (Though sometimes it's as simple as setting a `state=present` to `state=absent` for a few tasks and that's all that's required.
[1] https://www.ansiblefordevops.com
[2] https://www.ansibleforkubernetes.com
I'll add these links and your YouTube channel to our README now :).
The "it must be idempotent" argument lead to putting conditional into a a language that was never meant to be, doing the whole "store" danse that could be done with simple function calls and whole lot of conventions instead of a proof tested import system.
It's calling python code behind the scene anyway, and nothing reads the playbook apart from ansible itself.
Ansible is a great tool, but I strongly believe the experience would be 100% better without learning, crafting and debugging those twisted .yml.
It doesn't even really do that. It sets up a rube goldberg mechanism to push a python program containing a base64 encoded zip file containing the module it wants to execute it. Then it runs a shell to run python to run that module. This is why ansible is 2000x slower than it should be.
see https://github.com/ansible/ansible/issues/36275
https://mitogen.networkgenomics.com/
Getting rid of the yaml would make things faster, but compare to the cost of the network, I don't think it would be that much faster.
Ansible is used in the network automation space as well, but some people found they wanted more flexibility so they created 'raw' Python frameworks:
* https://nornir.readthedocs.io/en/latest/
* https://napalm-automation.net
Presentation:
* https://www.youtube.com/watch?v=3uIk0WQLHZk
Is it compatible with ansible?
Because ansible has one thing that can't be beaten: the ecosystem is huge. You have recipes for everything.
There's also a utility to help with the running of Ansible playbooks via Python (https://github.com/ansible/ansible-runner).
However, I've said this many times before—if your playbooks start mixing Python into tasks, you're either doing something wrong (it could be architected better), or it's time to write a custom Ansible module so the Python goes into a `.py` file and not into YAML.
99% of my playbooks are strict YAML with maybe a few Jinja filters, which make everything very readable. But I've also seen a ton of YAML that makes me want to barf.
See, if hello world is:
Then it would become: Instant benefits:- Python interpretter is very good at telling you error messages.
- You get an exception system, stack trace and a debugger for free
- The entire python tooling ecosystem, including full monthy IDE support, linters and formaters, is at your disposal
- Any more complex example is going to be much cleaner than the yaml version, exponentially. And no need to live in the "yaml/python module" dichotomy to keep PB clean.
- No need to reinvent the wheel. Python has elegant import, namespaces, ways to pass data around, conditionals, etc.,
- You can import Python libs. Event if it's just for declarative stuff like pendulum, appdirs..
The only cons are:
- idempotence needs to be enforced culturally. It's already the case for people writing the module behind the scene, and a good API + some warning tricks would be enough IMO.
- can't read the files outside of Python. But nobody does it anyway, and we can always let the Python files output the yaml representation if needed.
Now I don't hold my breath because it's a lot of work and the ansible community usually don't like the idea at all, and I'm not willing to do it. Fair is fair.
I was really excited about it, but unfortunately he stopped work on it since it wasn't getting much traction.
With Ansible, you can go _very_ far before you have to know anything about Python (well, most of the time—annoyingly, there are situations like naming groups where Python's variable syntax leaks through currently).
It would be like making people know basic Go syntax just to write or edit a Kubernetes manifest—that immediately limits its appeal to those who have at least a passing interest in Go (which is a much smaller audience than 'willing to learn YAML, which is used all over the place').
Anybody provisioning a machine knows a little bit of bash, and most probably python. Python is way more "used all over the place" than yaml.
Worst of all, when you are a beginner, ansible yaml errors are an utter mystery.
So how do people test their ansible playbooks if they can't easily reverse their state?
https://github.com/ansible-community/molecule
Ansible makes it easy to do mundane things like delete the default user and set up a new account, copy over ssh keys for easy access, add services and set up the docker container. Docker has the meaty stuff that takes ages to build and install, so I make sure it works and then save the image for easy download later.
What I've currently got going is everything in one repository, it has the playbooks, docker files, tests, service files, etc. I clone that using ansible on the target machine. Ansible then pulls the repo to get new changes and rebuilds the container. The container also adds that repo as a volume so that tests and other things are accessible. I don't store containers, I just spin up a new vm on boot.
I do have a "backup" playbook which installs everything inside the container when I want to test outside docker. This has happened a few times when using custom libraries that are specific to the embedded platform that don't come with the docker images (eg hardware accelerated gstreamer plugins come to mind). On the Pi life is a bit better because there are raspbian docker images anyway.
Molecule 3.x was recently released but there's been some major changes which "broke" my existing setup and I haven't yet gotten around to "fixing" it.
It takes a bit to learn how to use Molecule but, if you do much role development at all, it's definitely worth it!
Other times, I write a secondary playbook to roll back some deployment if needed. (Though sometimes it's as simple as setting a `state=present` to `state=absent` for a few tasks and that's all that's required.
If linux only, docker is great for that because it's so light, and it easier to compare layers of an image.