groodt

The article doesn’t provide any alternatives?

I think there are indeed many challenges when evaluating Compound AI Systems (http://bair.berkeley.edu/blog/2024/02/18/compound-ai-systems...)

But evals in complex systems are the best we have at the moment. It’s a “best-practice” just like all the forms of testing in the “test pyramid” (https://martinfowler.com/articles/practical-test-pyramid.htm...)

Nothing is a silver bullet. Just hard won, ideally automated, integrated quality and verification checks, built deep into the system and SDLC.

Is this a response to Chainguard? https://www.chainguard.dev/

Exciting!

I'm surprised to see nobody has mentioned gradio yet. It seems to sit in the same niche as streamlit and mesop.

https://gradio.app/

+1

Bazel and monorepo are a bit exotic to most enterprise Java devs, but now that I've experienced it, I'll never go back to Gradle or Maven or sbt.

Shopify and Target have been known to use it.

This is neat!

The nixos:latest reference is amusing. A reproducible image from a non-reproducible reference.

Thanks for publishing this. I quickly skimmed the paper, I saw the impressive linear scaling as you scaled to 16 nodes. How long did it take to train the various models in wall clock time?

Link to PEP 684 https://peps.python.org/pep-0684/

Any thoughts how to detect direct DML on the state table? Presumably allowing direct DML on the state table without the same on the Outbox table would lead to silent data corruption or lost updates.

What’s the story behind google3? What happened to google and google2?

Python packaging gets a lot of criticism. It's a meme. The thing is, it's actually improved dramatically over the years and continues to improve.

The problems it solves are very complex if one looks a little below the surface. It is solving different problems to the ecosystems that it's often compared to: golang, rust, java, js.

Does using this approach prevent direct DML changes or migrations outside application logic?

352 packages of top 360

Yes, the idea is a more efficient version of a Python zipapp https://realpython.com/python-zipapp/

It's called Python Packed Resources https://pyoxidizer.readthedocs.io/en/stable/oxidized_importe...

Looks like a very slick set of tools.

This is true.

However if the transitive-closure in the dependencies contains an sdist package anywhere in the tree, written in a way that wasn't designed to be imported on all platforms (this is rare, but does happen), then it isn't possible to resolve cross platform.

The prevalence of wheels does alleviate this to some extent. I think it is one of the early design decisions that pip can only install/resolve for the current platform and this will be very difficult to unwind. There are movements slowly in a direction that may one day make it possible to statically resolve python packages and remove dynamic dependency metadata or builds happening during dependency resolution.

342 out of 360

Data Platform Group lead at Canva here.

There are many open and commercial options for hyperparameter optimization.

We didn't select Argo for hyperparameter optimization. The situation is that because we already used Argo for our ML training and data processing jobs, we wanted to see if it could be extended for hyperparameter optimization. It actually can!

As mentioned briefly in the article, we try to use tools with minimal overlap. It's not always possible of course and sometimes new tools are necessary.