News for hackers. Stuff that matters.

savin-goyal

the topic of managing large dependency chains for ML/AI workloads in a reproducible has been a deep rabbit hole for us. if you are curious, here is some of the work in open domain

https://docs.metaflow.org/scaling/dependencies https://outerbounds.com/blog/containerize-with-fast-bakery

Re:350M Tokens Don't Lie: Love and Hate in Hacker News on Tuesday August 13, 2024 @09:00PM (#41241531)
what's up with the title flips from

> 350M Tokens Don't Lie: Love And Hate In Hacker News, to

> LLM-based sentiment analysis of Hacker News posts, to

> LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

Re:Maestro: Netflix's Workflow Orchestrator on Monday July 22, 2024 @05:10PM (#41039849)
Metaflow sits on top of Maestro, and neither replaces the other

> ...Users can use Metaflow library to create workflows in Maestro to execute DAGs consisting of arbitrary Python code. from https://netflixtechblog.com/orchestrating-data-ml-workflows-...

The orchestration section in this article (https://netflixtechblog.com/supporting-diverse-ml-systems-at...) goes into detail on how Metaflow interplays with Maestro (and Airflow, Argo Workflows & Step Functions)

Older »