News for hackers. Stuff that matters.

lwsio

Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @09:35AM (#49536010)
I will give it a look!
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @09:33AM (#49535995)
Coming back to this a few hours later, I've decided to add a section to explain this to the blog post. Thank you for flagging it.
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @09:27AM (#49535907)
I haven't, but that's just because it really isn't my personal usage pattern.
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @09:25AM (#49535866)
A literal manual typo. Good catch, will fix.
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @09:23AM (#49535841)
The beauty of this is that you can just swap out the platform and everything remains as it's the same backend. You make a really good point, one that I haven't really considered, but I also only have so many hours in the day to be fiddling around with stuff.
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @09:20AM (#49535808)
Hehe I really am just working it out as I go along - I promise it is fairly painless. Hugging Face allow you to specify your machine and then browse models that fit. And then you can just vibe out 'oh this one is a bit slow let me try another' etc
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @06:08AM (#49534180)
It is not
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @06:07AM (#49534170)
Running it depends on RAM, which is what I wrote, bandwidth is important for speed. I chose my words carefully, but you are absolutely right.
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @06:06AM (#49534161)
My perf sucks compared to yours. Added it to the post - same model averages 325 tok/s in processing prompts, and 34 tok/s in token generation. What am I doing wrong..?
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @06:05AM (#49534150)
I'm the author - hello! Added to the post! Qwen averages 325 tok/s in processing prompts, and 34 tok/s in token generation. That isn't instant, but it's quick enough that I never really think about it.
Re:My local model setup on an M4 Pro Mac Mini on Wednesday September 02, 2026 @06:05AM (#49534148)
I'm the author - hello! I talk about it in the blog post - knowing what's being run, knowing where it's being run, and not having anyone else control it.