News for hackers. Stuff that matters.
Re:ETL Pipelines with Airflow: The Good, the Bad and the Ugly by Arimbr on Friday October 08, 2021 @11:44AM (#28800513)
Nice work there! I also think that the next challenge for data teams is all this data documentation and discovery work.

I still think that Airflow is great for power data engineers. Airbyte and dbt are positioned to empower data analysts (or lazy data engineers like me) to own the ELTs.

Re:ETL Pipelines with Airflow: The Good, the Bad and the Ugly by rockostrich on Friday October 08, 2021 @12:27PM (#28801097)
Agreed. I see a lot of folks coming up with one off solutions for pulling data out of 3rd party sources like Kustomer or Lever. Giving a centralized UI for setting that up would be a great service.

Seems like I have a fun weekend project.

Re:ETL Pipelines with Airflow: The Good, the Bad and the Ugly by dominotw on Friday October 08, 2021 @11:35PM (#28806818)
what about meltano/singer