News for hackers. Stuff that matters.
Re:Airflow 2.0 by elephantum on Thursday December 17, 2020 @04:42PM (#25460859)
Thanks!

That's not exactly what I was looking for, though. Because every listed approach injects technical complexity in the middle of my business logic.

I.e. if I have two consequent tasks I have to define them as a separate scripts or commands, package them, upload and then orchestrate them in a completely different place.

While in Prefect I have all the niceness of writing almost plain Python (as with new tasks API in Airflow), then I can package and distribute the whole thing in docker image with a single command. It really matters!

Re:Airflow 2.0 by kumare3 on Thursday December 17, 2020 @11:58PM (#25464021)
Have you tried Flyte.org?
Re:Airflow 2.0 by elephantum on Friday December 18, 2020 @01:55AM (#25464529)
I did not, thank you for the tip!
Re:Airflow 2.0 by kumare3 on Friday December 18, 2020 @03:47PM (#25471755)
Ohh you are welcome, join the slack channel and ask for help. The community is growing everyday - here are some examples of using it in python https://flytecookbook.readthedocs.io/en/latest/