News for hackers. Stuff that matters.
Re:Nativ: Run frontier open models locally on your Mac by erikkahler on Monday July 20, 2026 @04:51PM (#48984710)
To clarify, this MIT-licensed app is from the very same dev, 'Prince Canuma', who maintains the popular MLX-VLM library (https://github.com/Blaizzy/mlx-vlm). MLX-VLM is a long-time dependency of the excellent LM Studio and others because it can provide faster inference on Apple devices than llama.cpp. Historically, MLX is a smaller community than CUDA, but has some of the fastest updates upon the release of new models, particularly in models with modalities beyond text-in, text-out (vision, STT, TTS, video gen). See also (https://github.com/Blaizzy/mlx-audio-swift). Would be totally unsurprised if those modalities and models get integrated into this UI.

Possibly vibe-coded landing page notwithstanding, the app is mostly written in Swift language. That suggests it will be easy to port this inference stack to iPad and iPhone.

Re:Nativ: Run frontier open models locally on your Mac by rahimnathwani on Monday July 20, 2026 @08:03PM (#48986493)

  Would be totally unsurprised if those modalities and models get integrated into this UI.
Yup, the GitHub repo says:

  Support for dedicated audio-only and image-generation-only models is coming soon.
Prince Canuma is super-responsive on X and GitHub issues, and I use mlx-audio almost daily with mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16 (for voice cloning).
Re:Nativ: Run frontier open models locally on your Mac by recursivegirth on Monday July 20, 2026 @07:58PM (#48986448)
LM Studio is trash on Windows / Linux... guess that makes sense...
Re:Nativ: Run frontier open models locally on your Mac by Forgeties79 on Tuesday July 21, 2026 @11:25AM (#48993530)
What makes it trash on Linux? I’m a pretty casual user myself so my guess is I haven’t bumped against these limitations. I also don’t have high expectations as I’m running it on a 9070 lol
Re:Nativ: Run frontier open models locally on your Mac by trollbridge on Monday July 20, 2026 @05:21PM (#48985024)
Thanks for that. My first question was “What does this do that Unsloth doesn’t?”
Re:Nativ: Run frontier open models locally on your Mac by carterschonwald on Monday July 20, 2026 @09:40PM (#48987094)
its a wrapper around mlx, so thats gonna be the portability bottleneck
Re:Nativ: Run frontier open models locally on your Mac by simonw on Monday July 20, 2026 @09:49PM (#48987147)
I was so excited when I saw "blaizzy" in the domain, because Prince Canuma's work around MLX has been of such uniquely high quality.
Re:Nativ: Run frontier open models locally on your Mac by justatdotin on Monday July 20, 2026 @11:11PM (#48987607)
Hi Simon, sorry to spam your comments but 7 months ago you asked me for a media report to back a claim I made and this week it finally arrived. perhaps a little late to be very useful to anyone who maintains that gating function, but here nonetheless:

https://ntindependent.com.au/scientist-says-ministers-pole-f...

Re:Nativ: Run frontier open models locally on your Mac by dostick on Tuesday July 21, 2026 @12:29PM (#48994567)
What
Re:Nativ: Run frontier open models locally on your Mac by walrus01 on Monday July 20, 2026 @10:35PM (#48987403)
I would also note that for people who want to download these models, you can find MLX versions of just about everything popular on huggingface these days. For instance go look at the "main" page for Qwen 3.6 35B-A3B and then follow the link to quantizations, and pick one of the more popular/reputable MLX variants.

https://huggingface.co/Qwen/Qwen3.6-35B-A3B

Re:Nativ: Run frontier open models locally on your Mac by sgt101 on Tuesday July 21, 2026 @12:06PM (#48994181)
converting a model to MLX is trivially easy as well - just a couple of lines of code and a few minutes wait.

I do find that it reliably crashes on the first try, but then it picks up from where it finishes and completes the conversion reliably.

Re:Nativ: Run frontier open models locally on your Mac by layla5alive on Tuesday July 21, 2026 @05:43AM (#48990064)
Prince CacheAsNUMA? Nice :)
Re:Nativ: Run frontier open models locally on your Mac by Der_Einzige on Tuesday July 21, 2026 @07:09AM (#48990705)
Switching to mlx-vlm is basically harmful since it (like vllm and sglang) have such garbage support for modern samplers. To be clear, I am one of the authors on the min_p paper, and if min_p is the best you have (when llamacpp supports the far superior top-n-sigma), than I have no reason to switch even if you are somehow faster.

(https://arxiv.org/abs/2411.07641)

And if you do care to support modern samplers, you can start with the following:

1. https://arxiv.org/abs/2509.23234

2. https://arxiv.org/abs/2509.02510

3. https://arxiv.org/abs/2604.11012

Re:Nativ: Run frontier open models locally on your Mac by woadwarrior01 on Tuesday July 21, 2026 @09:18AM (#48991935)
Thanks for bringing this up. You're 100% right. But most people, even technical ones are oblivious to how much of a difference modern samplers and higher quality quantization algorithms make for on-device LLM inference and are stuck with good old top-p, top-k samplers and RTN quantization.

TBF mlx-vlm does support min-p sampling, but none of the other modern samplers that you list. Ollama and LM Studio are even worse with only top-p and top-k samplers.