News for hackers. Stuff that matters.
Re:Airbnb Engineering by chuckcode on Thursday June 04, 2015 @06:45PM (#9662701)
Seems like it is getting to be a trend to write your own workflow engine and open source it. Spotify has luigi, LinkedIn has Azkaban and Airbnb has Airflow. I wonder at what market capitalization companies start rolling their own language like .net, go and swift?
Re:Airbnb Engineering by rattray on Friday June 05, 2015 @12:27AM (#9663901)
How is that in any way surprising? The more resources you have, the more custom-tailored you can make your tools. And I can't imagine it's a shock that data workflows aren't a completely-solved-for-all-usecases problem at this point...
Re:Airbnb Engineering by eva1984 on Thursday June 04, 2015 @07:11PM (#9662815)
Don't forget Chronos.

http://mesos.github.io/chronos/

Re:Airbnb Engineering by toomuchtodo on Thursday June 04, 2015 @09:12PM (#9663288)
Isn't Chronos more of a distributed scheduler ("fault tolerant cron") than a workflow engine?
Re:Airbnb Engineering by eva1984 on Thursday June 04, 2015 @09:56PM (#9663430)
Distributed job scheduler is, in a larger sense, a workflow engine. It will need to understand what is the next job to trigger, handle different situation where job fails\timeouts and do the retry if specified.
Re:Airbnb Engineering by samkone on Friday June 05, 2015 @07:04AM (#9664816)
which was also started at Airbnb.
Re:Airbnb Engineering by carrollgt on Thursday June 04, 2015 @08:12PM (#9663075)
drake[0] from factual falls into this category as well.

[0] - https://github.com/Factual/drake

Re:Airbnb Engineering by perlgeek on Friday June 05, 2015 @01:14AM (#9663985)
Airflow reminded me a lot of taskflow from the OpenStack project.
Re:Airbnb Engineering by firloop on Thursday June 04, 2015 @07:11PM (#9662811)
Well, Facebook spun PHP off into Hack if that counts.
Re:Airbnb Engineering by thejosh on Thursday June 04, 2015 @09:52PM (#9663413)
All a small framework called 'Ruby on Rails', released by Basecamp.
Re:Airbnb Engineering by kroosaidher on Friday June 05, 2015 @12:21AM (#9663885)
Hey, isn't a DSL your own language, strictly speaking? So any internal company-specific XML schema etc...