News for hackers. Stuff that matters.
Re:Nextflow: Data-Driven Computational Pipelines by esafak on Thursday August 10, 2023 @04:08PM (#37081094)
In all fairness, they predate the competition (2013): https://github.com/nextflow-io/nextflow/releases?page=25
Re:Nextflow: Data-Driven Computational Pipelines by geoffjentry on Thursday August 10, 2023 @04:26PM (#37081297)
As GP referenced CWL, while NF had appeared first in terms of the bioinformatics world Nextflow, CWL, Snakelike, and WDL all erupted close enough to each other to be equal-ish. The people were aware of each other but they were all so nascent that it wasn't clear if it was worth joining in or not. At the end of the day these all came from groups trying to scratch particular itches, and not everyone agreed on the right way to scratch.

However all of them were rejections of prior models as well as the workflow solutions prominent in the business space.

Re:Nextflow: Data-Driven Computational Pipelines by adolph on Thursday August 10, 2023 @04:56PM (#37081664)
Yeah, the thing that I find disappointing is that there is a lot of science value locked into the different systems of describing a workflow, pipeline or DAG. Like you said, they all had different itches to scratch and even some barebones "standards" like csv have flavors/extensions/etc.
Re:Nextflow: Data-Driven Computational Pipelines by bafe on Thursday August 10, 2023 @05:24PM (#37081983)
They try to address similar solutions, but comparing snakemake and nextflow doesn't do either tool a favour. They use different computation models, nextflow is based on dataflow programming and therefore schedules processes dynamically as new data comes in, while snakemake is pull-based and schedules the processes based on the dag defined by the dependencies. Anyhow they are both great tools.
Re:Nextflow: Data-Driven Computational Pipelines by geoffjentry on Thursday August 10, 2023 @07:26PM (#37083039)
While true that’s a minor distinction when comparing the clusters of bioinformatics workflow systems vs workflow systems aimed at different domains
Re:Nextflow: Data-Driven Computational Pipelines by bafe on Friday August 11, 2023 @01:34AM (#37085431)
While these two are aimed at bioinformatics, they are general purpose enough that you can apply them to any computational workflow. I can say they saved my PhD
Re:Nextflow: Data-Driven Computational Pipelines by mbreese on Thursday August 10, 2023 @06:38PM (#37082622)
In fairness, this is an old problem with many other contenders. This issue is as old as batch schedulers. FWIW, I was at an ISMB conference in 2005 that had at least 2-3 workflow managers presented.