News for hackers. Stuff that matters.
Re:Data Engineering Patterns with Apache Airflow [video] by caravel on Wednesday August 29, 2018 @03:27PM (#17871005)
For context, I used Datastage, Informatica, Ab Initio and SSIS in previous lives an went on to write the first version of Airflow. I developed a taste for pipelines-as-code while working at Facebook using an internal tool that is not open source.

I'd argue that pipelines as code, as opposed to dragndrop GUIs, is a better approach, at least for people who are comfortable writing code. Code is easy to version, test, diff, collaborate and allows for the creation of arbitrary abstractions.

There are no comments on this comment yet.