News for hackers. Stuff that matters.
Re:Data Engineering Patterns with Apache Airflow [video] by neuromantik8086 on Wednesday August 29, 2018 @03:23PM (#17870964)
This is why I wish attempts at standardizing ways to express DAGs in YAML/JSON/XML like CWL [0] and WDL [1] had more steam to them. If these standards took off, you'd be able to take your workflow and execute it on another batch system scheduler if you got tired of your current workflow orchestration tool.

[0] https://www.commonwl.org/ [1] https://software.broadinstitute.org/wdl/

There are no comments on this comment yet.