News for hackers. Stuff that matters.

A comment on Lessons learned from running Apache Airflow at scale. Parent comment

Re:Lessons learned from running Apache Airflow at scale by riku_iki on Monday May 23, 2022 @01:31PM (#31482105)
> already comfortable paying through the nose for trivial things you can run yourself for free

But fault tolerant workflow engine is not trivial thing, it may cost you many engineer hours to build, monitor and maintain it, so outsourcing it to someone else is totally viable solution.

[ Parent | on HN ]
Re:Lessons learned from running Apache Airflow at scale by arinlen on Monday May 23, 2022 @02:09PM (#31482775)
> But fault tolerant workflow engine is not trivial thing,

The complexity and risk of migrating cloud providers eclipses whatever problem you assign to "fault tolerant workflow engines".

Any mention of AWS Step Functions makes absolutely no sense at all and reads at best like a non-sequitur.

[ Parent | on HN ]
Re:Lessons learned from running Apache Airflow at scale by thorum on Monday May 23, 2022 @03:37PM (#31483916)
I read it as a comment on the UX / developer experience, which can superior with Step Functions vs competition regardless of whether Step Functions is an appropriate (or even physically possible) option for non-AWS projects.
[ Parent | on HN ]