Re:Lessons learned from running Apache Airflow at scale
by veeti on Monday May 23, 2022 @02:35PM
(#31483186)
FWIW if you don't need Airflow to catch up and backfill missed tasks, you can either set catchup=False on the DAG or use a LatestOnlyOperator.
[ Parent
| on HN ]
Re:Lessons learned from running Apache Airflow at scale
by subsaharancoder on Monday May 23, 2022 @06:46PM
(#31485804)
I have catchup=False set in the DAG but that hasn't stopped Airflow from back filling missed tasks, not sure why this is the case?