Vision Now Available in Llama.cpp

Re:Vision Now Available in Llama.cpp by ngxson on Saturday May 10, 2025 @02:51AM (#43943706)

We also support SmolVLM series which delivers light-speed response thanks to its mini size!

This is perfect for real-time home video surveillance system. That's one of the ideas for my next hobby project!

    llama-server -hf ggml-org/SmolVLM-Instruct-GGUF
    llama-server -hf ggml-org/SmolVLM-256M-Instruct-GGUF
    llama-server -hf ggml-org/SmolVLM-500M-Instruct-GGUF
    llama-server -hf ggml-org/SmolVLM2-2.2B-Instruct-GGUF
    llama-server -hf ggml-org/SmolVLM2-256M-Video-Instruct-GGUF
    llama-server -hf ggml-org/SmolVLM2-500M-Video-Instruct-GGUF

[ Parent | on HN ]

Re:Vision Now Available in Llama.cpp by a_e_k on Saturday May 10, 2025 @04:09AM (#43944005)

I've been noticing your commits as I skim the latest git commit notes whenever I periodically pull and rebuild. Thank you for all your work on this (and llama.cpp in general)!

[ Parent | on HN ]

Re:Vision Now Available in Llama.cpp by thatspartan on Saturday May 10, 2025 @07:50AM (#43944933)

Thanks for landing the mtmd functionality in the server. Like the other commenter I kept poring over commits in anticipation.

[ Parent | on HN ]

Re:Vision Now Available in Llama.cpp by moffkalast on Saturday May 10, 2025 @08:51AM (#43945225)

Ok but what's the quality of the high speed response? Can the sub-2.2B ones output a coherent sentence?

[ Parent | on HN ]