Airflow and the Future of Data Engineering: A Q&A

Re:Airflow and the Future of Data Engineering: A Q&A by cardosof on Wednesday March 01, 2017 @02:09AM (#13761889)

I'm used to running R scripts with cron to handle some batch jobs and I'm no python dev. Would it be easy to start using Airflow?

[ Parent | on HN ]

Re:Airflow and the Future of Data Engineering: A Q&A by smooc on Wednesday March 01, 2017 @03:00AM (#13762125)

DAGS in Airflow can just be a few lines. Some understanding of the syntax of python is required. But you can start simple and add complexity as you require it.

[ Parent | on HN ]

Re:Airflow and the Future of Data Engineering: A Q&A by classybull on Wednesday March 01, 2017 @08:10AM (#13763374)

To add to that, you can create a DAG class that's really just a wrapper that executes your R script and waits for its return code.

[ Parent | on HN ]