Re:Mara: A lightweight ETL framework, halfway between plain ...
by KirinDave on Wednesday May 09, 2018 @03:45PM
(#17033264)
Let me give you a quippy answer:
Cron is awful for cloud-based deployments trying to keep data from multiple sources in sync and compute business-specific metrics over them.
ETL frameworks seek to address this pain by building a large stack of software around scheduling the running of and unifying the writing of these data munging tasks.
As such, they tend to be as awful as the sum of their dependencies minus some small constant quality factor. So, more awful than any one part but less awful than the storm you'd be howling into if you didn't take some sort of infrastructural solution.
Cron is awful for cloud-based deployments trying to keep data from multiple sources in sync and compute business-specific metrics over them.
ETL frameworks seek to address this pain by building a large stack of software around scheduling the running of and unifying the writing of these data munging tasks.
As such, they tend to be as awful as the sum of their dependencies minus some small constant quality factor. So, more awful than any one part but less awful than the storm you'd be howling into if you didn't take some sort of infrastructural solution.