Re:Ask HN: How do you handle long-running workflows at your ...
by tedmiston on Monday May 14, 2018 @03:58PM
(#17068906)
Airflow can run tasks written in languages besides Python in several ways, such as through the BashOperator, DockerOperator, dispatching a job to a Spark cluster, etc. It's common to use multiple languages.
It's really just the configuration for tasks, DAGs, etc that must be done in Python. I know some people have even automating that to pull from yaml or json instead, but I prefer to have the flexibility myself.
It's really just the configuration for tasks, DAGs, etc that must be done in Python. I know some people have even automating that to pull from yaml or json instead, but I prefer to have the flexibility myself.