News for hackers. Stuff that matters.
Re:AWS Fargate – Run Containers Without Managing Infrastructure by andrewstuart2 on Wednesday November 29, 2017 @12:40PM (#15808946)
I know you're answering the question of bin-packing, but after two years of experience with it, I can say that for me, bin-packing is one of the smallest benefits (though it sells very well with management), though perhaps a baseline requirement these days. The real benefits, in my experience, stem from the declarative nature of cluster management, and the automation of doing the sensible thing to enact changes to that declarative desired state.
Re:AWS Fargate – Run Containers Without Managing Infrastructure by eropple on Wednesday November 29, 2017 @12:44PM (#15809000)
Sure. CloudFormation exists for that, though, and both its difficulty and its complexity are way overstated while also letting you manage AWS resources on top of that.

And it doesn't cost anything to use.

Re:AWS Fargate – Run Containers Without Managing Infrastructure by andrewstuart2 on Wednesday November 29, 2017 @12:58PM (#15809120)
Eh, there are a lot of terrible things I'd rather put myself through than writing another CloudFormation template for any sort of complex infrastructure. It could have been made easier and more readable if my company had allowed the use of something like Monsanto's generator [1], but creating ASTs in JSON is not my idea of a good user experience.

[1] https://github.com/MonsantoCo/cloudformation-template-genera...

Re:AWS Fargate – Run Containers Without Managing Infrastructure by eropple on Wednesday November 29, 2017 @02:08PM (#15809791)
I maintain auster[1] and am a contributor to cfer[2] for exactly that purpose. ;) CloudFormation really isn't a rough time anymore, IMO.

[1] - https://github.com/eropple/auster

[2] - https://github.com/seanedwards/cfer

Re:AWS Fargate – Run Containers Without Managing Infrastructure by dvlsg on Wednesday November 29, 2017 @03:29PM (#15810546)
If you know those tools exist, maybe. I just put together a new project using cloudformation (technically serverless, but it turned into 90 percent cloudformation syntax anyways), and it was pretty rough.
Re:AWS Fargate – Run Containers Without Managing Infrastructure by eropple on Wednesday November 29, 2017 @03:38PM (#15810616)
Maybe it's just me, but as a programmer the first thing I ever asked when looking at the wall of CloudFormation JSON was "so how do we make this not suck?".

Our job is not just to automate servers, it's to automate processes, including stupid developer-facing ones.

Re:AWS Fargate – Run Containers Without Managing Infrastructure by jrochkind1 on Wednesday November 29, 2017 @05:14PM (#15811395)
True, but as a _programmer_, working on a _new to me platform or package_, I am _very_ reluctant to add an extra third-party abstraction layer which requires it's own evaluation of quality and stability and some learning curve. It's gotta be pretty clear to me that it really is "what everyone else is doing", or I've gotta get more experience with the underlying thing to be able to judge for myself better.

I've definitely been burned many times by adding an extra tool or layer meant to make things easier, that ends up not, for all manner of reasons. I think we all have.

Re:AWS Fargate – Run Containers Without Managing Infrastructure by wgjordan on Wednesday November 29, 2017 @09:01PM (#15812831)
Worth noting that "nearly 31,000 AWS CloudFormation stacks were created for Prime Day" [1], so Amazon uses CloudFormation heavily internally. Not a guarantee that it's what 'everyone else is doing', but it's a good indicator of quality/stability and that it will remain a core service within the AWS ecosystem for some time.

[1] https://aws.amazon.com/blogs/aws/prime-day-2017-powered-by-a...

Re:AWS Fargate – Run Containers Without Managing Infrastructure by hayd on Wednesday November 29, 2017 @10:24PM (#15813217)
I think they're talking about a CF template generator being the third-party software (I could be wrong).
Re:AWS Fargate – Run Containers Without Managing Infrastructure by andrewstuart2 on Wednesday November 29, 2017 @03:51PM (#15810704)
You're not wrong. But in my case, that had been basically forbidden as an option, essentially because it "wasn't supported by Amazon," and because there's just additional risk to non-standard approaches. AWS certifications cover CloudFormation, so you can hire for that with low risk pretty easily. Other nonstandard utilities, not so much.
Re:AWS Fargate – Run Containers Without Managing Infrastructure by mr_toad on Wednesday November 29, 2017 @09:29PM (#15812983)
Cloudformation templates can be written in YAML now, which is a lot less sucky than writing JSON by hand.
Re:AWS Fargate – Run Containers Without Managing Infrastructure by wahnfrieden on Wednesday November 29, 2017 @04:31PM (#15811065)
If your only experience with CloudFormation is hand-written JSON, it's worth another look.

We used to use troposphere, a Python library for generating CloudFormation templates, but have since switched back to vanilla CloudFormation templates now that they added support for YAML. We're finding it's much nicer to read and write in plain YAML. We're also now using Sceptre for some advanced use cases (templatizing the templates, and fancier deployment automation).

Re:AWS Fargate – Run Containers Without Managing Infrastructure by sjellis on Thursday November 30, 2017 @02:00AM (#15813955)
> If your only experience with CloudFormation is hand-written JSON, it's worth another look.

Strongly agree.

YAML and sensible formatting conventions really do transform the usability of CloudFormation.

Re:AWS Fargate – Run Containers Without Managing Infrastructure by owaislone on Wednesday November 29, 2017 @03:34PM (#15810580)
And so does terraform which is pretty awesome!
Re:AWS Fargate – Run Containers Without Managing Infrastructure by eropple on Wednesday November 29, 2017 @03:48PM (#15810685)
Terraform requires significant infrastructure to get the same state management and implicit on-device access that CloudFormation's metadata service does. A common pattern in systems I oversee or consult on is to use CloudFormation's metadata service (which is not the EC2 metadata service, to be clear) to feed Ansible facts or chef-zero attributes in order to have bootstrapped systems that do not rely upon having a Tower or Chef Server in my environment.

The Terraform domain spec is not sufficiently expressive (just look at the circumlocutions you need to not create something in one environment versus another). It's way too hard to build large modules, but the lack of decent scoping makes assembling many small modules difficult too. Worse, the domain spec is also requires HCL, which is awful, or JSON, which is a return to the same problem that cfer solves for CloudFormation. One of my first attempts at a nontrivial open-source project was Terraframe[1], a Ruby DSL for Terraform; I abandoned it out of frustration when it became evident that Terraform's JSON parsing was untested, broken, and unusable in practice. Out of that frustration grew my own early CloudFormation prototypes, which my friend Sean did better with cfer.

If you're looking for an alternative to CloudFormation, I generally recommend BOSH[2], as it solves problems without introducing new ones. Saying the same for Terraform is a stretch.

[1] - https://github.com/eropple/terraframe

[2] - https://github.com/cloudfoundry/bosh

Re:AWS Fargate – Run Containers Without Managing Infrastructure by Rapzid on Wednesday November 29, 2017 @08:27PM (#15812626)
Cloudformation is not without its problems, even still. I feel you overstate Terraform's issues, though there is tons of valid criticism to go around. I would say Terraform really shines in areas CF still does not.

We use Terraform for our foundation and a clever Cloudformation custom resource hack to export values out to use in Cloudformation stacks where appropriate(use with Serverless services, etc). Works great for us; Terraform has seen significant(surprising even if you haven't looked at it in 6+ months) development over the past year.

Re:AWS Fargate – Run Containers Without Managing Infrastructure by ttul on Wednesday November 29, 2017 @01:08PM (#15809215)
Immutability, in other words.