Airflow and the Future of Data Engineering: A Q&A

Re:Airflow and the Future of Data Engineering: A Q&A by caravel on Wednesday March 01, 2017 @11:15AM (#13764905)

[author of Airflow here] as I wrote in another comment, I'd argue for a programmatic approach to workflows/dataflows as opposed to drag and drop. It turns out that code is a better abstraction for software: https://medium.freecodecamp.com/the-rise-of-the-data-enginee...

I'd also argue for open source over proprietary, mostly to allow for a framework that is "hackable" and extensible by nature. You can also count on the community to build a lot of the operators & hooks you'll need (Airflow terms).

[ Parent | on HN ]

Re:Airflow and the Future of Data Engineering: A Q&A by larsf on Wednesday March 01, 2017 @03:07PM (#13767748)

You can use Composable's Fluent API to author and run dataflows in code. No GUI required. Composable's platform is open and completely extensible in sense that anyone can add applications or first class modules to the system, including swapping out some of the more internal components.

[ Parent | on HN ]