News for hackers. Stuff that matters.
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by danielhanchen on Saturday November 14, 2026 @03:01AM (#50495028)
If anyone has any questions as well - feel free to ask!

We also updated https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF with UD3.1 quants

And also https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF as well!

We provide our benchmarking methods like what graders we use, infra fixes and issues and more in https://unsloth.ai/docs/basics/dynamic-3.0-ggufs/dynamic-3.1...

Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by rdslw on Saturday November 14, 2026 @04:00AM (#50495339)
how to best run your 27G v3.1 quants on recent mac hardware with 32-64gb of ram. mlx etc. ?

what would you recommend re: software stack for best pp/tg overall performance?

Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by danielhanchen on Saturday November 14, 2026 @04:05AM (#50495356)
Oh llama.cpp has massively improved so it should function well! We're also making MLX faster soon!
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by rdslw on Saturday November 14, 2026 @10:06AM (#50497603)
look forward it. thank you for your work on unsloth!
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by regexorcist on Saturday November 14, 2026 @11:36AM (#50498361)
Check out kapitsa for mac.
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by logicalmind on Saturday November 14, 2026 @01:23PM (#50499402)
If you're looking to run something from the Qwen family, you should jump right to Splash. It's faster than mlx or llama. You can run the unsloth gguf's.

https://github.com/incoai/splash

Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by dr_kiszonka on Saturday November 14, 2026 @04:32AM (#50495502)
Hi! Unsloth Studio is great, and I was wondering where exactly you are with multi-GPU and multi-node training and fine-tuning? I have been checking your documentation and GitHub issues from time to time and there doesn't seem to be as much development as I initially anticipated. Is making this aspect as user friendly as other parts of Studio and the docs on your near term roadmap or are your prioritizing other areas for now?
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by danielhanchen on Saturday November 14, 2026 @04:51AM (#50495613)
Oh we have not announced it, but in the past month, Unsloth has FSDP2 support - full finetuning, LoRA and QLoRA all work! We also support pipeline parallel, tensor parallel, and more!

See https://github.com/unslothai/notebooks/blob/main/fsdp2_scrip...

Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by mirod1 on Saturday November 14, 2026 @04:41AM (#50495551)
In order to run the model, I need to install the Unsloth desktop, on Ubuntu. The link at https://unsloth.ai/download/linux says it should be 44.3MB, but the download itself is 23M. Is this an oversight on the download page or am I getting the wrong package?
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by danielhanchen on Saturday November 14, 2026 @04:49AM (#50495601)
Oh it's correct - that's the installer size - but we will need to download binaries from https://github.com/unslothai/llama.cpp and also PyTorch etc which will take 2-3GB more
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by dofm on Saturday November 14, 2026 @09:12AM (#50497174)
Is this new technique limited to Qwen-type architectures, or can e.g. Muse Glimmer be updated this way?
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by danielhanchen on Sunday November 15, 2026 @01:14AM (#50504449)
Oh sorry for the delay - oh UD3.1 was utilized for DeepSeek-V4.1, Qwen3.8-Flash-Next and Qwen3.8-27B :) But yes it's generalized! We're investigating Gemma-4 next! And yes other models are in the pipeline!

Oh PDF export - hm I will check :) No worries!

Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by dofm on Sunday November 15, 2026 @02:25PM (#50509833)
Cool. I still really like Gemma 4 12B QAT — I actually think in many ways that's 2026's most important LLM.
Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update by doctorpangloss on Saturday November 14, 2026 @12:09PM (#50498646)
How is your company doing? Why not ask Gemini Carbon to quantize for a workload?