Re:Experimenting with Local LLMs on macOS
by DrAwdeOccarim on Monday September 08, 2025 @06:52PM
(#45175163)
I adore Qwen 3 30b a3b 2507. Pretty easy to write an MCP to let us search the web with Brave API key. I run it on my Macbook Pro M3 Pro 36 GB.
Re:Experimenting with Local LLMs on macOS
by theshrike79 on Tuesday September 09, 2025 @07:40AM
(#45180576)
What are you running it on that lets you connect tools to it?
Re:Experimenting with Local LLMs on macOS
by DrAwdeOccarim on Tuesday September 09, 2025 @08:57AM
(#45181329)
LM Studio. I just vibe code the nodeJS code.
Re:Experimenting with Local LLMs on macOS
by balder1991 on Tuesday September 09, 2025 @01:05AM
(#45177620)
You’ll certainly find better answers on /r/LocalLlama in Reddit for this.
Re:Experimenting with Local LLMs on macOS
by frontsideair on Tuesday September 09, 2025 @02:34AM
(#45178232)
This is the command probably:
sudo sysctl iogpu.wired_limit_mb=184320
Source: https://github.com/ggml-org/llama.cpp/discussions/15396
* General Q&A
* Specific to programming - mostly Python and Go.
I forgot the command now, but I did run a command that allowed MacOS to allocate and use maybe 28 GB of RAM to the GPU for use with LLMs.