Re:Mara: A lightweight ETL framework, halfway between plain ...
by martin_loetzsch on Wednesday May 09, 2018 @02:08PM
(#17032260)
(author here).
Mara data integration is indeed a glorified version of Make (with cost based scheduling and lots of visualizations)
The ETL frameworks (Airflow, Luigi, now Mara) help with this, allowing you to build dependency graphs in code, determine which dependencies are already satisfied, and process those which are not. They'll usually contain helper code for common ETL tasks, such as interacting with a database, writing to/reading from S3, or running shell scripts.