Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Show HN: Warmctl – Fast cold starts for vLLM inference.
I’m building this GPU checkpoint & restore tool to optimize vLLM cold start time for GPU neoclouds
It’s dead simple at UX:
- Run warmctl build once in your CI/CD
- warmctl push the warm image to any OCI registry
- warmctl up on every new replica on GPU hosts
That’s it, so you can serve even Deepseek 4.1 or GLM 5.3 < 60 seconds
There are no comments on this comment yet.
I’m building this GPU checkpoint & restore tool to optimize vLLM cold start time for GPU neoclouds
It’s dead simple at UX:
- Run warmctl build once in your CI/CD
- warmctl push the warm image to any OCI registry
- warmctl up on every new replica on GPU hosts
That’s it, so you can serve even Deepseek 4.1 or GLM 5.3 < 60 seconds