Re:Show HN: Deepinfra.com Serverless AI model hosting (top m...
by julien_c on Wednesday February 22, 2023 @06:40AM
(#34894286)
quite cool! haven't tried it yet, but what's the latency on hot-loading a model? (for instance, loading `stabilityai/stable-diffusion-2-1` for the first API call)
[ Parent
| on HN ]
Re:Show HN: Deepinfra.com Serverless AI model hosting (top m...
by nikola_borisof on Friday February 24, 2023 @02:46PM
(#34929102)
Because this is popular model and many people use it, you will not experience the cold-start latency most likely. But in general it is <10s.