News for hackers. Stuff that matters.

A comment on Lessons learned from running Apache Airflow at scale. Parent comment

Re:Lessons learned from running Apache Airflow at scale by carschno on Monday May 23, 2022 @12:11PM (#31480876)
Do you have more examples for better tools, ideally open source (unlike AWS Step functions)?
[ Parent | on HN ]
Re:Lessons learned from running Apache Airflow at scale by gadflyinyoureye on Monday May 23, 2022 @01:08PM (#31481757)
Flowable is a BPNM system. You can do a lot of async calls with it. https://www.flowable.com/open-source

We use it for a complex pricing process that invokes 30-40 micro services that can take up to minutes per step.

[ Parent | on HN ]
Re:Lessons learned from running Apache Airflow at scale by qw on Monday May 23, 2022 @01:29PM (#31482068)
Kamelets and the Karavan UI in combination with k8s and Knative for "serverless" integrations looks interesting.
[ Parent | on HN ]
Re:Lessons learned from running Apache Airflow at scale by jonpon on Monday May 23, 2022 @03:38PM (#31483928)
We at magniv.io are building an alternative.

Our core is open source https://github.com/MagnivOrg/magniv-core

We can set you up with our hosted if you would like to poke around!

[ Parent | on HN ]
Re:Lessons learned from running Apache Airflow at scale by antupis on Monday May 23, 2022 @02:28PM (#31483094)
We are using prefect+dbt and I like it. Altought they are doing huge rewrite at the moment.
[ Parent | on HN ]