Re:Dabbling with Dagster vs. Airflow
by jmcnevin on Wednesday September 14, 2022 @02:36PM
(#32841352)
I'd say give Temporal (https://temporal.io) a look, but there are a lot of options (https://github.com/meirwah/awesome-workflow-engines).
[ Parent
| on HN ]
Re:Dabbling with Dagster vs. Airflow
by skrtskrt on Wednesday September 14, 2022 @03:24PM
(#32842045)
Seconding Temporal, it is definitely a second-gen workflow platform with approaches that make many Airflow frustrations nonexistent.
In particular, workflows are extremely durable, they can easily just sit indefinitely waiting on a condition and you can redeploy workers whenever, and workflows will resume exactly where they were.
Re:Dabbling with Dagster vs. Airflow
by valenterry on Wednesday September 14, 2022 @04:11PM
(#32842713)
Wow, that looks cool. Especially since the way of defining things in code looks pretty first-class.