Re:AWS Managed Workflows for Apache Airflow
by kingkonz1 on Tuesday November 24, 2020 @04:46PM
(#25203318)
not OP, but I prefer https://github.com/azkaban/azkaban to airflow
[ Parent
| on HN ]
Re:AWS Managed Workflows for Apache Airflow
by zmmmmm on Tuesday November 24, 2020 @05:04PM
(#25203477)
so, workflows described in yaml? I also am a bit ambivalent about Airflow and the way workflows are built up in it, but YAML seems really weak and unlikely to handle complexity well.
Also - it's entirely built around Hadoop from what I can see? Seems a limited use case compared to Airflow.
Re:AWS Managed Workflows for Apache Airflow
by ttymck on Tuesday November 24, 2020 @05:06PM
(#25203507)
You raise a good point, there is an apparent need for an application that retains the dags-as-code but radically simplifies the airflow architecture.
Re:AWS Managed Workflows for Apache Airflow
by rywalker on Tuesday November 24, 2020 @05:15PM
(#25203610)
if you like YAML-based DAGs, you can do that in Airflow with the dag-factory extension https://github.com/ajbosco/dag-factory