News for hackers. Stuff that matters.
Re:Introduction to Apache Airflow by tiagod on Friday May 29, 2020 @10:20AM (#23350245)
I absolutely love Cadence/Temporal. It can do anything the others can do (and you could implement any other engine with it), without any DSLs.
Re:Introduction to Apache Airflow by bjacobso on Friday May 29, 2020 @12:23PM (#23351969)
I'd also like to +1 on Cadence/Temporal. It's a really great mental model and framework for thinking about workflows.

This is more of a proof of concept but it can also support DSLs (although we found the go client is easier to understand than DSLs): https://github.com/checkr/states-language-cadence

Re:Introduction to Apache Airflow by mfateev on Friday May 29, 2020 @01:07PM (#23352568)
As a team lead of the Cadence/Temporal it is really amazing to hear such a feedback. Thank you!

BTW. You can AMA about the technology here.

Re:Introduction to Apache Airflow by ttsda on Friday May 29, 2020 @01:30PM (#23352882)
Thank you for your work! I believe once it's properly documented and marketed, it will be very popular.
Re:Introduction to Apache Airflow by cpdean on Friday May 29, 2020 @10:47AM (#23350523)
I think this is the link to cadence: https://github.com/uber/cadence

and temporal: https://github.com/temporalio/temporal

Re:Introduction to Apache Airflow by tiagod on Friday May 29, 2020 @10:59AM (#23350683)
Yep, this is it. They're developed by the same team, Temporal is still beta but I believe the production version is coming out late-June.

I can't find an easy way to explain everything it does, but it pretty much allows you to write naive functions with no error handling, with month-long sleeps, auto-retries on unreliable function calls, etc etc.

It also gives you a web interface when you can inspect the running functions, and allows for external code (and other workflow functions) to signal/query the running workflows.

Re:Introduction to Apache Airflow by silasb on Friday May 29, 2020 @10:29AM (#23350321)
Just found out about Temporal and it looks interesting. I eager to jump in but our organization primarily uses Ruby. I know the big difference between Cadence vs Temporal is the fact they are using GRPC which seems much easier to adopt.
Re:Introduction to Apache Airflow by ttsda on Friday May 29, 2020 @11:00AM (#23350692)
I also find the API to be more consistent on Temporal (I began implementing a project in Cadence before finding out about Temporal)
Re:Introduction to Apache Airflow by mfateev on Friday May 29, 2020 @11:29AM (#23351107)
Temporal has an externally contributed Ruby client. It is going to be open sourced very soon.
Re:Introduction to Apache Airflow by silasb on Friday May 29, 2020 @11:44AM (#23351335)
Very exciting news. Is there a way that I can subscribe to get info on when it'll be released?
Re:Introduction to Apache Airflow by mfateev on Friday May 29, 2020 @12:00PM (#23351585)
There is #ruby-sdk channel in the Temporal Slack: https://join.slack.com/t/temporalio/shared_invite/zt-c1e99p8...
Re:Introduction to Apache Airflow by autogenerated on Friday May 29, 2020 @12:02PM (#23351618)
Moving off of Airflow and to Cadence/Temporal was the single biggest relief in terms of maintainability, operational ease and scalability. Also +1 on being free of any DSL.
Re:Introduction to Apache Airflow by ttsda on Friday May 29, 2020 @12:39PM (#23352167)
I'm currently moving from a custom yaml DSL-based engine to Temporal and it's the best architectural decision I've taken in a long time. I researched a lot and couldn't find anything that even came close to the freedom it provides.
Re:Introduction to Apache Airflow by alexg123 on Friday May 29, 2020 @07:49PM (#23356997)
Curious about this. Can you elaborate more? Also happy to hop on a call/zoom if you don't want to share publicly (email me at [email protected]). I'm working on something similar.