News for hackers. Stuff that matters.

ericcurtin

Re:Cloudflare Down Again – and DownDetector Is Also Down on Friday December 05, 2025 @04:02AM (#46158420)
Time to use some local ai with Docker Model Runner :)

No cloudflare no problem

https://github.com/docker/model-runner

I'm one of the devs, happy to answer any questions

Show HN: Docker Model Runner Integrates vLLM for High-Throughput Inference

https://github.com/docker/model-runner

Hi HN, I’m one of the authors of this post.We’ve updated Docker Model Runner to support vLLM alongside the existing llama.cpp backend. The goal is to bridge the gap between local prototyping (often done with GGUF/llama.cpp) and high-throughput production (often done with Safetensors/vLLM) using a consistent Docker workflow.Key technical details:Auto-routing: The tool detects the model format....

( Read More... | 1 comment | 5 points | on HN )
Re:Show HN: Run Unsloth Dynamic GGUFs using Docker model runner on Wednesday November 19, 2025 @02:11PM (#45983724)
Will be around the next few hours

Show HN: Run Unsloth Dynamic GGUFs using Docker model runner

https://github.com/docker/model-runner

We've been working with Docker to make running quantized models easier with Docker. This integration allows users to pull and run Unsloth-optimized models using the native Docker CLI.Key features:Uses docker model run syntax.Supports Dynamic GGUFs (handling quantization overhead).Cross-platform support.Example usage: docker model run ai/gpt-oss:20BHappy to answer questions about the...

( Read More... | 1 comment | 2 points | on HN )
Hi everyone, we're the maintainers.

We're rebooting the model-runner community and wanted to share what we've been up to and where we're headed.

When we first built this, the idea was simple: make running local models as easy as running containers. You get a consistent interface to download and run models from different backends (llama.cpp being a key one) and can even transport them using familiar OCI registries like Docker Hub.

Recently, we've invested a lot of effort into making it a true community project. A few highlights:

- The project is now a monorepo, making it much easier for new contributors to find their way around.

- We've added Vulkan support to open things up for AMD and other non-NVIDIA GPUs.

- We made sure we have day-0 support for the latest NVIDIA DGX hardware.

- There is a new "docker model run" UX: https://www.docker.com/blog/docker-model-run-prompt/

Show HN: Docker/Model-Runner – Join Us in Revitalizing the DMR Community

https://github.com/docker/model-runner

Hey Hacker News,We're the maintainers of docker/model-runner and wanted to share some major updates we're excited about.Link: https://github.com/docker/model-runnerWe are revitalizing the community:https://www.docker.com/blog/revitalizing-model-runner-commun...At its core, model-runner is a simple, backend-agnostic tool for downloading and running local large language models. Think of it as a...

( Read More... | 2 comments | 13 points | on HN )
Keep opening pull requests and issues, we need these things, you are right!
Any new features you think we should add to further enhance your usage? Glad you find it useful
Is there overlap with "Show HN: docker/model-runner – an open-source tool for local LLMs" posted today, we encapsulate AI workloads in containers using docker model runner. Some clear encapsulations advantages at a minimum. Could we work together?
What did you like? Anything stand out?
Thanks very much. It worked well for you? Which hardware? :) Any other feedback, keep it coming!
Hi everyone, we're the maintainers.

We're rebooting the model-runner community and wanted to share what we've been up to and where we're headed.

When we first built this, the idea was simple: make running local models as easy as running containers. You get a consistent interface to download and run models from different backends (llama.cpp being a key one) and can even transport them using familiar OCI registries like Docker Hub.

Recently, we've invested a lot of effort into making it a true community project. A few highlights:

- The project is now a monorepo, making it much easier for new contributors to find their way around.

- We've added Vulkan support to open things up for AMD and other non-NVIDIA GPUs.

- We made sure we have day-0 support for the latest NVIDIA DGX hardware.

Show HN: docker/model-runner – an open-source tool for local LLMs

https://github.com/docker/model-runner

Hey Hacker News,We're the maintainers of docker/model-runner and wanted to share some major updates we're excited about.Link: https://github.com/docker/model-runnerWe are rebooting the community:https://www.docker.com/blog/rebooting-model-runner-community...At its core, model-runner is a simple, backend-agnostic tool for downloading and running local large language models. Think of it as a...

( Read More... | 10 comments | 16 points | on HN )