Etsy is under pressure to start acting more like a conventional company

Re:Etsy is under pressure to start acting more like a conven... by giobox on Thursday May 18, 2017 @12:57PM (#14368874)

I completely see how it's theoretically desirable to not be tied to a particular cloud/VPS provider, but I'm not entirely convinced Terraform alone is the answer. In my experience, Terraform does not let you declare VPS resources in a "provider neutral" way - you would still have to completely rewrite your Terraform files if you say, switch from AWS to Azure. Then there's the wider issue that whole concepts in one VPS provider that you used in Terraform don't necessarily exist in another, or require a completely different approach. A great example is just look at the documentation for Terraform's DigitalOcean Support, vs their AWS support. The AWS documentation is many times larger! Once your stack reaches a certain degree of complexity this is a non-trivial task. I should make clear this isn't a knock against Terraform - it's a great tool, but it isn't going to magically make it easy to leave AWS for lots of people.

Amazon aren't stupid, and to really realise the full value/potential of their cloud offering you kind of have to design for the proprietary features of their stack, which of course is the whole point - driving you to be locked in as much as they can. Treat them as just a bunch of generic compute boxes in the cloud and you will end up driving your costs up, not to mention often markedly increasing the complexity of your deployment.

[ Parent | on HN ]

Re:Etsy is under pressure to start acting more like a conven... by eropple on Thursday May 18, 2017 @01:09PM (#14368984)

You touch on what I think is a bigger concern, as somebody who spends a lot of time on this sort of work: Terraform doesn't actually solve any problem that I encounter. You aren't "provider neutral" when using it, as you say, and moving from one nontrivial provider (AWS, GCE, Azure) to another is a full rewrite. I disagree--and rather strongly at that--that it's a lock-in thing, unless you are aggressively climbing all over the proprietary offerings from each cloud. Even the basics of the stack are different in ways that are pretty annoying to deal with (just trying to port simple autoscaling groups and load balancers is work--not hard work, but work, and work for which you must write code that must be tested). Attempts to encapsulate general functionality (and I've been suckered into this myself) fail because the abstractions are super leaky. It's not worth the fight.

We have a lot of clients who say they want to be multi-cloud, they say they're afraid of being locked into AWS or GCE, and in practice nobody except the supermassive companies out there ever actually do move between them. To that end, we pretty much standardized on using the platform-specific options: CloudFormation, Azure Resource Manager, and Google Cloud Deployment Manager.

(There is a small argument to some of the ancillary services that Terraform can wire up, but to be honest I rarely find a compelling reason to put up with Terraform and its lovely habit of fragging my state in order to wire those up declaratively rather than using a little imperative glue within Auster[0] to do it.)

[0] - https://github.com/eropple/auster

[ Parent | on HN ]

Re:Etsy is under pressure to start acting more like a conven... by giobox on Thursday May 18, 2017 @02:25PM (#14369730)

I think Terraform does solve some problems, for some teams. For us it's really the process it provides - by moving VPS infrastructure definition to code, rather than management UIs, we can apply our existing code review processes we have to our hardware infrastructure before promoting through staging environments, not just the product code. This is vastly preferable to letting individual engineers run wild at the AWS management console, and allows for rollback of breaking changes much more easily etc. This arguably the real value in Terraform, which i think earlier posts about using it to somehow become cloud neutral missed.

[ Parent | on HN ]

Re:Etsy is under pressure to start acting more like a conven... by eropple on Thursday May 18, 2017 @02:49PM (#14369934)

Sure. I feel very strongly that Terraform is strictly inferior to CloudFormation in doing this. Terraform breaks its own promises (JSON support was broken for a long time, I'm not even sure it's fully there now), has demonstrated regressions in releases (going so far as to break existing states by doing so), and has enough of a habit of horking existing states that multiple clients separately coined the term "terrafucked" for having their states trashed and necessitating manual intervention.

CloudFormation isn't perfect--but it isn't a hand grenade.

[ Parent | on HN ]