msinxi
Re:ETL Pipelines with Airflow: The Good, the Bad and the Ugly
on Friday October 08, 2021 @06:38PM (#28804930)
Checkout great expectations (https://greatexpectations.io/). You basically run assertions on your data. It auto-generates documentation for you and you can also store results of your validations in a flat file or database. I think there are dbt modules that try to mimic great expectations' validation but I like the docs that come with great expectations