News for hackers. Stuff that matters.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by JSR_FDED on Friday June 26, 2026 @11:38PM (#48694896)
MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly.

Dedicated GPUs have less video RAM so can run smaller less smart models quickly.

Re:Ask HN: MacBook vs. Dedicated GPU for LLM by exabrial on Friday June 26, 2026 @11:57PM (#48694993)
Do Mac Pros provide more headroom? noob here, noob questions
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by rho138 on Saturday June 27, 2026 @12:17AM (#48695086)
Idk why you’re being downvoted for asking a question. Pending specs they _could_ provide more headroom for a larger model but they would still be limited by the CPU and it’s associated bus speeds.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by JSR_FDED on Saturday June 27, 2026 @12:52AM (#48695239)
In what sense? Headroom for what?
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by zihotki on Saturday June 27, 2026 @12:58PM (#48699828)
Not that much - a bit better but still negligible. Prefill is highly parallelizable and benefits from many cores. NVidia GPU's simply have several times more cores than even Pros.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by mzubairtahir on Saturday June 27, 2026 @12:24AM (#48695118)
how much memory is actually useable by gpu in macbook? as it is shared?
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by pylotlight on Saturday June 27, 2026 @12:39AM (#48695185)
roughly ~50–56GB, although this is somewhat configurable with iogpu.wired_limit_mb. By default, macOS reserves ~25% of memory for the system.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by visarga on Saturday June 27, 2026 @12:28AM (#48695139)
Macbook M5 64GB - can run gemma-4-26b-a4b-it-4bit and Qwen3.6-35B-A3B-4bit at about 1500 tps prefix and 45 tps decode on contexts up to 100K tokens using MLX. It's faster than Claude. I was really surprised, chat quality is also similar to Claude for gemma4. Agentic works but does not compare to cloud models, you can still make agents where top level is code.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by mzubairtahir on Saturday June 27, 2026 @12:30AM (#48695148)
sorry but asking again: how much memory is actually useable by gpu in macbook? as it is shared(os and apps also have to use same memory)? and it is different than dedicated gpu memory?
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by gizajob on Saturday June 27, 2026 @12:40AM (#48695193)
It’s completely shared so the OS and everything else takes up maybe 8GB of the RAM. On a 64GB machine you can run models about 45GB in size and still have space for those models run other tasks which themselves might need ram. To a user, the GPU appears to just use the RAM as much as it needs same as any other process running on the system. You can see what space your LLMs are taking up in Activity Monitor (or htop) and how much GPU capacity they’re using (all of it)
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by cco on Saturday June 27, 2026 @03:06AM (#48695892)
Rule of thumb is about 70-80%.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by j45 on Saturday June 27, 2026 @12:59AM (#48695278)
You can adjust the percentage available both on the MacOS side and how much the model uses.
Re:Ask HN: MacBook vs. Dedicated GPU for LLM by EagnaIonat on Saturday June 27, 2026 @01:43AM (#48695507)
> MacBooks with their unified memory behave like a slow GPU with enormous amount of video RAM. So you can run large smart models slowly.

With the model using MLX the speed increase is night and day. Even non-MLX is good.

You also don't have the transfer costs related to moving CPU data into the GPU.

Re:Ask HN: MacBook vs. Dedicated GPU for LLM by emmettbt on Monday June 29, 2026 @08:39PM (#48727186)
Unified memory is also a kicker for crashing a Mac.