News for hackers. Stuff that matters.
Re:Show HN: Deepinfra.com Serverless AI model hosting (top m... by julien_c on Wednesday February 22, 2023 @06:40AM (#34894286)
quite cool! haven't tried it yet, but what's the latency on hot-loading a model? (for instance, loading `stabilityai/stable-diffusion-2-1` for the first API call)
Re:Show HN: Deepinfra.com Serverless AI model hosting (top m... by nikola_borisof on Friday February 24, 2023 @02:46PM (#34929102)
Because this is popular model and many people use it, you will not experience the cold-start latency most likely. But in general it is <10s.