News for hackers. Stuff that matters.
Re:Lessons learned from running Apache Airflow at scale by veeti on Monday May 23, 2022 @02:35PM (#31483186)
FWIW if you don't need Airflow to catch up and backfill missed tasks, you can either set catchup=False on the DAG or use a LatestOnlyOperator.
Re:Lessons learned from running Apache Airflow at scale by subsaharancoder on Monday May 23, 2022 @06:46PM (#31485804)
I have catchup=False set in the DAG but that hasn't stopped Airflow from back filling missed tasks, not sure why this is the case?