Re:Common Workflow Language
by prpl on Friday November 15, 2019 @08:24AM
(#21544534)
The attempt to cleanly separate out workflow is nice in theory but an extremely leaky abstraction when it comes to real world operations for DAGs larger than 10s of nodes. You will end up with lots of code that will duplicate functionality that exists in a workflow engine just to have something common (e.g. notifications, database ingest, etc...) There are no operational concerns in CWL.
[ Parent
| on HN ]
That’s my main issue with CWL. It’s a nice execution language, but not something I like to write an analysis pipeline in. Because it is YAML, using runtime logic is difficult (impossible?). I much rather use a pipeline tool that includes some flow control and ability to adapt to different arguments. Last I looked at CWL, this wasn’t the case.
Adding conditional execution to the dataflow graph is something being worked on for the next version of CWL.