News for hackers. Stuff that matters.

A comment on Llama.cpp Now Supports Qwen2-VL (Vision Language Model). Parent comment

Re:Llama.cpp Now Supports Qwen2-VL (Vision Language Model) by therein on Sunday December 15, 2024 @01:00AM (#42421814)
Are we at a point where we could run nonquantized models from QwQ/Qwen series on a 128GB Macbook Pro?
[ Parent | on HN ]
Re:Llama.cpp Now Supports Qwen2-VL (Vision Language Model) by simonw on Sunday December 15, 2024 @01:13AM (#42421857)
I think so. Are bf16 models nonquantized? There's an MLX one here that should fit on that machine: https://huggingface.co/mlx-community/QwQ-32B-Preview-bf16
[ Parent | on HN ]