Re:AWS Managed Workflows for Apache Airflow
by vanpythonista on Tuesday November 24, 2020 @07:28PM
(#25204643)
I like Argo and have used both solutions, but I'm not yet convinced that YAML based workflows are superior to Airflow's Python Code as workflows.
[ Parent
| on HN ]
Re:AWS Managed Workflows for Apache Airflow
by seddonm1 on Tuesday November 24, 2020 @09:16PM
(#25205278)
Agree that YAML is not really ideal for this case.
For other readers Argo does support 'Scripts' which means you can implement complex logic similar to Airflow if required but can be a bit clumsy: https://argoproj.github.io/argo/examples/#scripts-results
Additionally, I think we may see other layers on top of Argo in future that may abstract from the YAML similar to how Kubeflow uses Argo for the Kubeflow Pipelines capability: https://www.kubeflow.org/docs/components/pipelines/pipelines...