10k+ GPUs for Inference Jobs (waitlist giveaway)

Posted by bobjmiles on Tuesday December 27, 2022 @09:29AM
from the salad.com dept.

https://salad.com/salad-inference-endpoints

( 3 comments | 3 points | on HN )

Re:10k+ GPUs for Inference Jobs (waitlist giveaway) by boxerbk on Tuesday December 27, 2022 @09:41AM (#34149053)

Do you have an easy way to see the cost differential between this and public cloud alternatives?

[ Parent | on HN ]

Re:10k+ GPUs for Inference Jobs (waitlist giveaway) by bobjmiles on Tuesday December 27, 2022 @09:57AM (#34149188)

Only BERT (running on CPU only instances) and Stable Diffusion (on RTX3090) for now :) https://salad.com/salad-cloud/generative-ai

[ Parent | on HN ]

Re:10k+ GPUs for Inference Jobs (waitlist giveaway) by bobjmiles on Tuesday December 27, 2022 @09:29AM (#34148933)

Salad just launched a waitlist for their new Inference API running atop distributed infrastructure, 10k's of GPUs available.

The network is built from consumer nodes and comes at a fraction the cost of any other provider. - 6x inferences per dollar for Stable Diffusion (3090) - 4x inf/dollar for BERT (CPUs)

$1k/$100 in credits available on the waitlist: salad.com/salad-inference-endpoints

----- Salad also has a fully managed container service in beta, accessible at portal.salad.com, but account verification is required to get a container quota >0 (you go through the full setup and then prompted for more info)

[ Parent | on HN ]