News for hackers. Stuff that matters.

A comment on Metaflow, Netflix's Python framework for data science, is now open source. Parent comment

Re:Metaflow, Netflix's Python framework for data science, is... by sandGorgon on Wednesday December 04, 2019 @04:38AM (#21700841)
sorry- i meant notebook figuratively.

so metaflow is the local dev version of workflow construct. and then export that to airflow/etc compatible format ?

what workflow engine do you guys use and primarily support in metaflow ?

[ Parent | on HN ]
Re:Metaflow, Netflix's Python framework for data science, is... by vtuulos on Wednesday December 04, 2019 @04:43AM (#21700867)
the workflow DAG is a core concept of Metaflow but Metaflow is not just that. Other core parts are managing and inspecting state, cloud integration, and organizing results.

At Netflix, we use an internal workflow engine called Meson https://www.youtube.com/watch?v=0R58_tx7azY

[ Parent | on HN ]