hrichardlee

Hi HN,

This is a blog post + python code that demos how to run text to image ML models like Craiyon (formerly DALL·E Mini) yourself on AWS GPU compute, with the help of an OSS tool called Meadowrun we've been working on.

Happy to take any feedback or answer any questions!

Fair point, I've edited the Medium post's title, but I'm not able to make changes to the Hacker News post

Seems like we're definitely on the same wavelength! We'd love to hear more about your experiences--I'll reach out directly

Thank you!

Fargate is more of a maybe for us as it doesn't seem to offer a ton of advantages over EC2. It still takes about 30 seconds to launch a Fargate job, and as far as I can tell there's no way to "keep an instance around". With Meadowrun-on-EC2 or Lambda, when you run two jobs run one after another with the same libraries and the same code (or even slightly different code), there's almost 0 overhead for running the second job. So Fargate is only slightly better for a cold start (30s compared to 45-60s for an EC2 instance in my experience), and significantly worse for a warm start (still 30s). And that's the core experience we're trying to make amazing--run some code, look at the results/data, tweak it, run it again, repeat.

Meadowrun is taking care of all the messy details of provisioning and managing the EC2 instances, so Meadowrun-on-Fargate won't be any easier to use than Meadowrun-on-EC2, and I don't see a ton of advantages to make up for the inability to get a warm start on Fargate. That said, AWS is super dynamic, so we're definitely keeping an eye on Fargate.

Thank you! We appreciate it!

One pro is less overhead--I think all of the major options for an autoscaling Kubernetes cluster require at least one node that's always on. Meadowrun uses AWS Lambda/Azure Functions to manage instances, so it gets a lot closer to truly scaling down to zero.

Another pro is if your workflows aren't already container-based, not running on Kubernetes means we can build your containers for you on Meadowrun so you don't need to e.g. install Docker locally to get your libraries/code running on Meadowrun (it's hard to build containers in Kubernetes itself).

I mentioned this in another comment, but this also means we can e.g. use AWS Lambda as the compute layer, or if you have software that's hard to containerize, you can even use a custom AMI. (Both of these are features on the roadmap, so this is a bit theoretical at this point.)

The biggest con is probably that a lot of people already use Kubernetes, especially if they have an on-prem/hybrid deployment, or maybe if they have services with e.g. a load balancer that interact with their ad-hoc/batch jobs.

We are planning on adding the ability for Meadowrun to target Kubernetes as well, so Kubernetes takes care of the resource scheduling, but you still get the benefits of Meadowrun--a really simple API for running ad-hoc/batch jobs.