Re:Nextflow: Data-Driven Computational Pipelines
by getoffmycase on Friday August 11, 2023 @05:49PM
(#37094471)
Iād be happy to chat offline. Your current work and my current work I think share a lot of similarity. My GitHub is avcarr2.
To answer your broader question about the general need for some structured pipeline or workflow orchestration.. That comes down to volume of data (we do screens as well as one-off studies) and a desire to reduce human involvement as much as possible. So the goal is to have raw files be immediately picked up, processed, and loaded into on internal application where it can be queried and interesting data can be highlighted. During my PhD, this was also a goal of mine (and I have at least two github repos where I got close) but it was definitely less of a priority since actually doing experiments and downstream analysis was the limiting factor.
PS: if you want to talk off-HN, I should be your latest stargazer