Re:Ask HN: Who uses open LLMs and coding assistants locally?...
by amonroe805-2 on Friday October 31, 2025 @02:08PM
(#45774912)
Quick correction: The mac mini does have a fan. Studio is definitely more capable due to bigger, better chips, but my understanding is the mini is generally not at risk of thermal throttling with the chips you can buy it with. The decision for desktop macs really just comes down to how much chip you want to pay for.
Re:Ask HN: Who uses open LLMs and coding assistants locally?...
by embedding-shape on Friday October 31, 2025 @02:06PM
(#45774895)
> I do think Macs are phenomenal at running local LLMs if you get the right one.
How does the prompt processing speed look like today? I think it was either M3 or M4 together with 128GB, trying to run even slightly longer prompts took forever for the initial prompt processing so whatever speed gain you get at inference, basically didn't matter. Maybe it works better today?
Re:Ask HN: Who uses open LLMs and coding assistants locally?...
by giancarlostoro on Friday October 31, 2025 @08:54PM
(#45778334)
I have only ever used the M4 (on my wife's Macbook Air) and M4 Pro (on my Macbook Pro) and it was reasonable speeds, I was able to tie LM Studio with PyCharm and ask it questions about code, but my context Window kept running out, I don't think the 24GB model is the right choice, the key thing you have to also look out for is for example I might hvae 24GB of RAM, but only 16 of it can be used as VRAM, so I'm more competitive than my 3080 in terms of VRAM, though my 3080 could probably run circles around my M4 Pro if it wanted to.
Re:Ask HN: Who uses open LLMs and coding assistants locally?...
by Terretta on Friday October 31, 2025 @02:17PM
(#45775019)
And yes, for context windows / cached context, the MacBook Pro with 128GB memory is a mind boggling laptop.
The Studio Ultras are surprisingly strong as well for a pretty monitor stand.
I have a MacBook Pro with an M4 Pro chip, and 24GB of RAM, I believe only 16 of it is usable by the models, so I can run the GPT OSS 20B model (iirc) but the smaller one. It can do a bit, but the context window fills up quickly, so I do find myself switching context windows often enough. I do wonder if a maxed out MacBook Pro would be able to run larger context windows, then I would easily be able to code all day with it offline.
I do think Macs are phenomenal at running local LLMs if you get the right one.