News for hackers. Stuff that matters.
Re:AWS Batch – Fully Managed Batch Processing by jdoliner on Thursday December 01, 2016 @03:55PM (#13083424)
AdRoll has done some very cool stuff using containers for data processing. Afaik they were some of the first people to use this approach, and I think it's worked out quite well for them and given them quite an edge over their competitors.

If you're interested in building this type of infrastructure for yourself, I'd recommend taking a look at http://github.com/pachyderm/pachyderm. I'm one of the founders so this is a somewhat shameless plug, but Pachyderm is designed to get people most of the way to what AdRoll has built without having to build as much of the "plumbing" from scratch. AdRoll was some of our earliest inspiration for the design of Pachyderm.

Re:AWS Batch – Fully Managed Batch Processing by vtuulos on Thursday December 01, 2016 @04:08PM (#13083516)
Thanks! Pachyderm's version control over datasets is a killer feature.