News for hackers. Stuff that matters.

A comment on $500 GPU outperforms Claude Sonnet on coding benchmarks. Parent comment

Re:$500 GPU outperforms Claude Sonnet on coding benchmarks by 0xbadcafebee on Thursday March 26, 2026 @11:48PM (#47538943)
No? You can run any model that fits in its VRAM, and you can run larger models with layer/MoE offloading. Ask an AI what the best models you can run on that card are, then ask it for newer models than that. Ask what tuning options to pass to llama.cpp, and what the auto-tuning options are. Use ROCm builds.

It looks like your card has 16GB VRAM? Start with Qwen 3.5 9B Unsloth GGUFs (UD-Q6_K_XL) and branch out from there.

[ Parent | on HN ]
Re:$500 GPU outperforms Claude Sonnet on coding benchmarks by metalliqaz on Friday March 27, 2026 @02:24PM (#47546399)
I've been running local models on my 9070XT and I have never found ROCm to be faster than Vulkan
[ Parent | on HN ]
Re:$500 GPU outperforms Claude Sonnet on coding benchmarks by 0xbadcafebee on Friday March 27, 2026 @05:01PM (#47548211)
It's not guaranteed to be faster, but it is faster in some cases due to ROCm taking more advantage of specific chipset features
[ Parent | on HN ]