News for hackers. Stuff that matters.
Re:Airflow 2.0 by eobropta on Monday December 21, 2020 @01:43PM (#25497963)
I recommend checking out Conducto. It handles dependencies well by leveraging Docker and it is very easy to get started. It can also handle super complicated pipelines that required scaled up processing by just changing one input on the command line.

There are no comments on this comment yet.