News for hackers. Stuff that matters.

Show HN: Run Unsloth Dynamic GGUFs using Docker model runner

https://github.com/docker/model-runner

We've been working with Docker to make running quantized models easier with Docker. This integration allows users to pull and run Unsloth-optimized models using the native Docker CLI.

Key features:

Uses docker model run syntax.

Supports Dynamic GGUFs (handling quantization overhead).

Cross-platform support.

Example usage: docker model run ai/gpt-oss:20B

Happy to answer questions about the implementation or the quantization methods used.

https://docs.unsloth.ai/models/how-to-run-llms-with-docker

( 1 comment | 2 points | on HN )
Re:Show HN: Run Unsloth Dynamic GGUFs using Docker model runner by ericcurtin on Wednesday November 19, 2025 @02:11PM (#45983724)
Will be around the next few hours