News for hackers. Stuff that matters.
Re:AWS Batch – Fully Managed Batch Processing by vtuulos on Thursday December 01, 2016 @04:15PM (#13083568)
First time I hear about Elasticluster. Regarding cluster provisioning, auto-scaling is critical for cost and convenience. We managed to do this with a simple job queue that outputs a CloudWatch metric that in turn controls an auto-scaling group of spot instances. Works like a charm but AWS Batch should make this redundant.

We took a look at Airflow, which looks nice, but we haven't had a compelling reason not to use Luigi.

There are no comments on this comment yet.