Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Introduction to Apache Airflow. Parent comment
Yes of course, I mean Airflow is not a good fit for the tasks you don't want to be idempotent (I think most but not all tasks should be idempotent).
> I've never ran into issues with cross dag dependencies
I believe Airflow docs advice against them when possible. I see why from my experience: less visibility and more complexity, especially for backfills.
There are no comments on this comment yet.
Yes of course, I mean Airflow is not a good fit for the tasks you don't want to be idempotent (I think most but not all tasks should be idempotent).
> I've never ran into issues with cross dag dependencies
I believe Airflow docs advice against them when possible. I see why from my experience: less visibility and more complexity, especially for backfills.