Re:Habitat — A new approach to automation
by jacques_chester on Tuesday June 14, 2016 @11:17AM
(#11902508)
Comments based on the most trivial skimming of the docs, so I'm probably wrong about everything.
> Incredibly ambitious.
It is, but ambition is necessary if you want to build distributed production systems. BOSH[0] does something similar in providing an all-or-nothing approach to configuring distributed systems built out of heterogenous software components spread across multiple machines.
Note here that the key is the heterogenous part -- if you have an undifferentiated flock of software, you can rely on a different abstraction to deploy them (anything from PaaS to a hand-rolled thing, though I don't recommend hand-rolling this stuff). But very few distributed systems are designed to rely on emergent behaviour. In fact that's usually what we're trying to avoid.
Where they appear to differ is that BOSH is quite strict about lifecycle -- you can't "truly" update configurations on-the-fly. You update the relevant YAML and then ask BOSH to update the world for you. It seems like a hassle, but if you couple it with git and Concourse pipelines, you get a complete, externalised history of every system configuration change you've ever made.
We already know that this top-down model of heterogenous distributed systems configuration management works better than trying to extend it from the machine up (as Cfengine and its various successors tried to do). BOSH has been deploying large systems for years and years. It was originally designed to deploy and upgrade Cloud Foundry without downtime, something it does with aplomb.
What I really like about Habitat over BOSH is the way it exposes the available knobs and dials. That kind of interactive discoverability is super helpful sometimes.
I was, however, struck by this claim:
> This architecture allows us to avoid the production cliff problem: the closer you get to production, the more difficult it is.
This is literally the problem that PaaSes solve for you. I've seen companies without PaaSes, with home-rolled PaaSes and with Cloud Foundry. In the latter case, the biggest obstacle is usually getting VPN access. And there are other PaaSes apart from CF. Habitat seems to be doing a kinda-sorta PaaS here (note the "Supervisor"), but without most of the feature plumbing that makes PaaSes operationally useful: routing, logging, service injection and the like.
Disclaimer: I work for Pivotal, we do the majority of the engineering on both BOSH and Cloud Foundry.
> Incredibly ambitious.
It is, but ambition is necessary if you want to build distributed production systems. BOSH[0] does something similar in providing an all-or-nothing approach to configuring distributed systems built out of heterogenous software components spread across multiple machines.
Note here that the key is the heterogenous part -- if you have an undifferentiated flock of software, you can rely on a different abstraction to deploy them (anything from PaaS to a hand-rolled thing, though I don't recommend hand-rolling this stuff). But very few distributed systems are designed to rely on emergent behaviour. In fact that's usually what we're trying to avoid.
Where they appear to differ is that BOSH is quite strict about lifecycle -- you can't "truly" update configurations on-the-fly. You update the relevant YAML and then ask BOSH to update the world for you. It seems like a hassle, but if you couple it with git and Concourse pipelines, you get a complete, externalised history of every system configuration change you've ever made.
We already know that this top-down model of heterogenous distributed systems configuration management works better than trying to extend it from the machine up (as Cfengine and its various successors tried to do). BOSH has been deploying large systems for years and years. It was originally designed to deploy and upgrade Cloud Foundry without downtime, something it does with aplomb.
What I really like about Habitat over BOSH is the way it exposes the available knobs and dials. That kind of interactive discoverability is super helpful sometimes.
I was, however, struck by this claim:
> This architecture allows us to avoid the production cliff problem: the closer you get to production, the more difficult it is.
This is literally the problem that PaaSes solve for you. I've seen companies without PaaSes, with home-rolled PaaSes and with Cloud Foundry. In the latter case, the biggest obstacle is usually getting VPN access. And there are other PaaSes apart from CF. Habitat seems to be doing a kinda-sorta PaaS here (note the "Supervisor"), but without most of the feature plumbing that makes PaaSes operationally useful: routing, logging, service injection and the like.
Disclaimer: I work for Pivotal, we do the majority of the engineering on both BOSH and Cloud Foundry.
[0] https://bosh.io/