bobjmiles

I'm the Founder here at salad.com and we're starting to see customers use our network of GPUs for data processing pipelines. - web scraping - transcription and audio data processing - translation - etc, basically any data processing pipeline.

This was written by one of our SA's and is a very cool demo of using an LLM to retrieve and format data.

salad.com is great, but it only really makes sense if you have a GPU

HN Hey!

At salad.com we're launching a GPU Cloud that runs atop consumer hardware. Great for inference/tuning, not great for training (because of networking, vram restrictions, etc).

We've built a fully managed container service so you can easily deploy replicas at massive scale and we just ran a Stable Diffusion demo: churned out 9 million images in 24hrs across 750 GPUs (100+ images/second and 5000 images/$). Architecture and specifics here salad.com/resources/gpu-benchmark-stable-diffusion

We're thinking our next run will be to customise the SD models (various checkpoints & LoRAs) to demo something closer to real-world.

We'll also run whisper-large at massive scale to demo a million hours of transcription, and get a benchmark price on this.

Perhaps webscraping/common-crawl at scale with GPU processing at the edge: building datasets for LLM training?

Question for the HN community, what would you run?

We'd love your suggestions, and perhaps hand over the keys to see what you can do with 1000's of GPUs!

Bob - Salad Founder

It's no good for training, only inference, but there's salad.com that has 10k's of DAU with GPUs and they've built a managed container service, with a inference API coming soon https://salad.com/salad-inference-endpoints

Only BERT (running on CPU only instances) and Stable Diffusion (on RTX3090) for now :) https://salad.com/salad-cloud/generative-ai

Salad just launched a waitlist for their new Inference API running atop distributed infrastructure, 10k's of GPUs available.

The network is built from consumer nodes and comes at a fraction the cost of any other provider. - 6x inferences per dollar for Stable Diffusion (3090) - 4x inf/dollar for BERT (CPUs)

$1k/$100 in credits available on the waitlist: salad.com/salad-inference-endpoints

----- Salad also has a fully managed container service in beta, accessible at portal.salad.com, but account verification is required to get a container quota >0 (you go through the full setup and then prompted for more info)

The tens of thousands of GPUs on salad.com would love to power this :) Shoot us an email bob@