News for hackers. Stuff that matters.

A comment on Qwen3-Coder: Agentic coding in the world. Parent comment

Re:Qwen3-Coder: Agentic coding in the world by jychang on Wednesday July 23, 2025 @06:47AM (#44657704)
What cluster do you have to do the quantizing? I'm guessing you're not using a single machine with a 3090 in your garage.
[ Parent | on HN ]
Re:Qwen3-Coder: Agentic coding in the world by danielhanchen on Wednesday July 23, 2025 @09:28AM (#44659028)
Oh definitely not! I use some spot cloud instances!
[ Parent | on HN ]
Re:Qwen3-Coder: Agentic coding in the world by sleight42 on Wednesday July 23, 2025 @05:02PM (#44664014)
But you can get one of these quantized models to run effectively on a 3090?

If so, I'd love detailed instructions.

The guide you posted earlier goes over my (and likely many others') head!

[ Parent | on HN ]
Re:Qwen3-Coder: Agentic coding in the world by danielhanchen on Wednesday July 23, 2025 @07:27PM (#44665143)
Oh yes definitely! Oh wait is the guide too long / wordy? This section https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locall... shows how to run it on a 3090
[ Parent | on HN ]
Re:Qwen3-Coder: Agentic coding in the world by sleight42 on Thursday July 24, 2025 @06:06PM (#44676889)
Kind of you to respond! Thanks!

I have pretty bad ADHD. And I've only run locally using kobold; dilettante at DIY AI.

So, yeah, I'm a bit lost in it.

[ Parent | on HN ]
Re:Qwen3-Coder: Agentic coding in the world by danielhanchen on Thursday July 24, 2025 @06:45PM (#44677223)
Oh sorry - for Kobold - I think it uses llama.cpp behind the hood? I think Kobold has some guides on using custom GGUFs
[ Parent | on HN ]