Yeah the issue is once a sensor fires once, it doesn't reset and keep firing on new data

My homegrown solution is a sensor at the beginning, and at the end an airflow api call to trigger a dag run of the same dag. Not DagRunOperator because then no dag would never finish due to infinite recursion

It seems kinda sketchy so I'm considering a lower level Celery implementation or even GenStage