Qwen 3.6 27B is the sweet spot for local development

Re:Qwen 3.6 27B is the sweet spot for local development by thewebguyd on Monday June 29, 2026 @01:57PM (#48722706)

I'd go for at least 32GB+. It'll fit in 24GB but leaves you little to no room for context, and that's at 4-bit quantization.

If you want to run unquantized, you definitely need 128GB.

[ Parent | on HN ]

Re:Qwen 3.6 27B is the sweet spot for local development by Catloafdev on Monday June 29, 2026 @02:01PM (#48722777)

Nobody runs unquantized, there's literally no reason to. Q8 would be the largest anyone actually runs on consumer hardware for inference.

[ Parent | on HN ]

Re:Qwen 3.6 27B is the sweet spot for local development by bityard on Monday June 29, 2026 @05:15PM (#48725334)

Halving the precision of the weights is not a free lunch...

[ Parent | on HN ]

Re:Qwen 3.6 27B is the sweet spot for local development by Catloafdev on Monday June 29, 2026 @07:30PM (#48726685)

Q8 is virtually lossless. The quantization is much more noticeable around Q4 and below. FP16->Q8 on consumer hardware is 2x the speed at ~99.99% the quality.

[ Parent | on HN ]

Re:Qwen 3.6 27B is the sweet spot for local development by rvba on Tuesday June 30, 2026 @04:00AM (#48729743)

Any source that confirms the 99.99% quality?

[ Parent | on HN ]

Re:Qwen 3.6 27B is the sweet spot for local development by Catloafdev on Tuesday June 30, 2026 @07:55PM (#48740773)

I don't have a 'source' off-hand but I recommend reading up on it if you want to learn more. A lot of models on HF show a card demonstrating the different quality trade-offs between quants.

[ Parent | on HN ]

Re:Qwen 3.6 27B is the sweet spot for local development by bitexploder on Monday June 29, 2026 @02:43PM (#48723377)

It also comes down to inference speed, not "can I run this". 8-bit quant is quite a bit slower on an M5 Pro.

[ Parent | on HN ]