troiskaer
Re:Launch HN: Sematic (YC S22) – Open-source framework to build ML pip...
on Wednesday August 10, 2022 @07:15PM (#32419646)
Interesting approach on passing data between steps and constructing the overall graph - it will be interesting to see what the take rate is between the two approaches (of sematic and metaflow). On the UI front, Metaflow generates viz for all objects by default in @card; but how does Sematic package up PyTorch referenced in the example (https://docs.sematic.dev/real-example) for execution on the cloud? IIRC, Metaflow packages the cwd (in addition to @conda, @pip etc.) and relies on existing packages for local execution?
Edit: Digging deeper, Sematic relies on Bazel (https://docs.sematic.dev/execution-modes#dependency-packagin...) and needs a BUILD file to specify all the dependencies for cloud execution. It seems that the entire pipeline will execute as a single (or multiple) k8s pod(s) using the same environment?
I am quite interested in trying out Sematic. Any guidelines on what kind of scale Sematic can support today (and the near future)?
Re:Launch HN: Sematic (YC S22) – Open-source framework to build ML pip...
on Wednesday August 10, 2022 @03:50PM (#32417481)
How does Sematic compare to Metaflow? it optimizes for many of the same goals of Sematic - local workflows, cloud access, lineage tracking, state transfer etc?
Re:Real World Recommendation System
on Monday April 11, 2022 @05:43PM (#30995476)
What's your argument for Twitter/Uber/AirBnB being indisputably more technologically influential than Netflix? And let's please talk facts rather than opinions.
Re:Real World Recommendation System
on Monday April 11, 2022 @04:59PM (#30994942)
This is pretty much what both Netflix and Spotify do. I would argue that there isn't a canonical recommendations stack that FAANG is converging towards, and that's a direct corollary of differing business requirements and organizational structure.
Re:Real World Recommendation System
on Monday April 11, 2022 @04:50PM (#30994820)
> But none of them are particularly important
Nowhere was the argument made that somehow Netflix was more influential than Twitter/Uber/AirBnB, but your counter-argument that somehow it's less influential because you haven't heard of/used some projects directly holds no ground.
Re:Real World Recommendation System
on Monday April 11, 2022 @04:00PM (#30994020)
as well as to ML - Netflix Prize (https://en.wikipedia.org/wiki/Netflix_Prize) and Metaflow (https://github.com/Netflix/metaflow)
Re:Kedro – Creating reproducible, maintainable and modular data scienc...
on Saturday January 22, 2022 @10:12PM (#30043261)
thanks for the unnecessary advertisement.
Re:Kedro – Creating reproducible, maintainable and modular data scienc...
on Saturday January 22, 2022 @01:43PM (#30039552)
Seems like Kedro has a similar thesis to Metaflow - I will look into it.
Re:Kedro – Creating reproducible, maintainable and modular data scienc...
on Saturday January 22, 2022 @01:01PM (#30039119)
How does Kedro compare to MLFlow and Metaflow?