News for hackers. Stuff that matters.
Re:Lessons learned from running Apache Airflow at scale by Mayzie on Monday May 23, 2022 @01:33PM (#31482131)
The biggest pain point in Airflow I have experienced is the horrible and completely lacking documentation. The community support (Slack) won't (or can't) help with anything beyond basic DAG writing.

That sore point makes running and using the software needlessly frustrating, and honestly I won't ever be using it again because of it.

Re:Lessons learned from running Apache Airflow at scale by jlaneve on Monday May 23, 2022 @02:01PM (#31482620)
That’s one of the things we’re working on at Astronomer - check out the Astronomer Registry! registry.astronomer.io
Re:Lessons learned from running Apache Airflow at scale by idomi on Monday May 23, 2022 @01:47PM (#31482377)
Make sure to checkout ploomber, our support is seamless tons of docs (https://docs.ploomber.io/) and we take our users seriously. P.S. We integrate with airflow and other orchestrators if you still need to tackle those.
Re:Lessons learned from running Apache Airflow at scale by 8589934591 on Monday May 23, 2022 @01:47PM (#31482376)
I agree with this. The slack is just the core developers discussing further development and tickets. The documentation is lacking big time. The only response to this is to raise PRs to improve docs.
Re:Lessons learned from running Apache Airflow at scale by pid-1 on Monday May 23, 2022 @05:49PM (#31485206)
Agreed, I just would like to add the documentation got a lot better in the past couple of years.
Re:Lessons learned from running Apache Airflow at scale by artwr on Monday May 23, 2022 @02:59PM (#31483502)
Can I ask more about your use case that you could not find an answer for?