News for hackers. Stuff that matters.

A comment on Show HN: Data load tool(dlt)-Python library to automate the creation of datasets. Parent comment

Re:Show HN: Data load tool(dlt)-Python library to automate t... by adrianbr on Wednesday October 25, 2023 @07:54AM (#38011806)
There are multiple ways to run together - we will show a few in a demo coming out soon.

We also consider a tighter integration like with Airflow described here as a possible next step https://dlthub.com/docs/walkthroughs/deploy-a-pipeline/deplo...

We will investigate the interest incrementally as to not build any plugins that don't end up used.

[ Parent | on HN ]
Re:Show HN: Data load tool(dlt)-Python library to automate t... by nerdponx on Wednesday October 25, 2023 @01:49PM (#38015791)
For an example of prior art, you should look into Astronomer's Cosmos Library to see how they integrate Dbt into Airflow.
[ Parent | on HN ]
Re:Show HN: Data load tool(dlt)-Python library to automate t... by adrianbr on Wednesday October 25, 2023 @02:28PM (#38016388)
Thank you! That's the example we looked at for our dlt-airflow integration :) the dlt dag becomes an airflow dag.
[ Parent | on HN ]