News for hackers. Stuff that matters.
Re:Dabbling with Dagster vs. Airflow by jmcnevin on Wednesday September 14, 2022 @02:36PM (#32841352)
I'd say give Temporal (https://temporal.io) a look, but there are a lot of options (https://github.com/meirwah/awesome-workflow-engines).
Re:Dabbling with Dagster vs. Airflow by skrtskrt on Wednesday September 14, 2022 @03:24PM (#32842045)
Seconding Temporal, it is definitely a second-gen workflow platform with approaches that make many Airflow frustrations nonexistent.

In particular, workflows are extremely durable, they can easily just sit indefinitely waiting on a condition and you can redeploy workers whenever, and workflows will resume exactly where they were.

Re:Dabbling with Dagster vs. Airflow by valenterry on Wednesday September 14, 2022 @04:11PM (#32842713)
Wow, that looks cool. Especially since the way of defining things in code looks pretty first-class.