News for hackers. Stuff that matters.
Re:Laptop Isn't Ready for LLMs. That's About to Change by fancyfredbot on Tuesday December 23, 2025 @03:56PM (#46369365)
I think the author is aware of Apple silicon. The article mentions the fact Apple has unified memory and that this is advantageous for running LLMs.
Re:Laptop Isn't Ready for LLMs. That's About to Change by dangus on Tuesday December 23, 2025 @03:58PM (#46369388)
Then idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory.

> However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero.

This straight up isn’t true.

Re:Laptop Isn't Ready for LLMs. That's About to Change by andai on Tuesday December 23, 2025 @04:17PM (#46369567)
So I'm hearing a lot of people running LLMs on Apple hardware. But is there actually anything useful you can run? Does it run at a usable speed? And is it worth the cost? Because the last time I checked the answer to all three questions appeared to be no.

Though maybe it depends on what you're doing? (Although if you're doing something simple like embeddings, then you don't need the Apple hardware in the first place.)

Re:Laptop Isn't Ready for LLMs. That's About to Change by seanmcdirmid on Tuesday December 23, 2025 @06:54PM (#46370915)
I have an MBP Max M3 with 64GB of RAM, and I can run a lot at useful speed (LLMs run fine, diffusion image models run OK although not as fast as they would on a 3090). My laptop isn't typical though, it isn't a standard MBP with a normal or pro processor.
Re:Laptop Isn't Ready for LLMs. That's About to Change by fhsm on Tuesday December 23, 2025 @05:43PM (#46370363)
This paper shows a use case running on Apple silicon that’s theoretically valuable:

https://pmc.ncbi.nlm.nih.gov/articles/PMC12067846/

Who cares if result is right / wrong etc as it will all be different in a year … just interesting to see a test of desktop class hardware go ok.

Re:Laptop Isn't Ready for LLMs. That's About to Change by jki275 on Tuesday December 23, 2025 @05:35PM (#46370282)
I can definitely write code with a local model like Devstral small or a quantized granite, or a quantized deep-seek on an M1 Max w/ 64gb of ram.
Re:Laptop Isn't Ready for LLMs. That's About to Change by DANmode on Tuesday December 23, 2025 @05:06PM (#46370034)
Of course it depends what you’re doing.

Do you work offline often?

Essential.

Re:Laptop Isn't Ready for LLMs. That's About to Change by sueders101 on Tuesday December 23, 2025 @09:50PM (#46371926)
I've tried out gpt-oss:20b on a MacBook Air (via Ollama) with 24GB of RAM. In my experience it's output is comparable to what you'd get out of older models and the openAI benchmarks seem accurate https://openai.com/index/introducing-gpt-oss/ . Definitely a usable speed. Not instant, but ~5 tokens per second of output if I had to guess.
Re:Laptop Isn't Ready for LLMs. That's About to Change by anonzzzies on Tuesday December 23, 2025 @09:59PM (#46371960)
I was sitting in an airplane next to a guy on a MacBook pro something who was coding in cursor with a local llm. We got talking and he said there are obviously differences but for his style of 'English coding' (he described basically what code to write/files to change but in english, but more sloppy than code obviously otherwise he would just code) it works really well. And indeed that's what he could demo. The model (which was the OSS gpt i believe) did pretty well in his nextjs project and fast too.
Re:Laptop Isn't Ready for LLMs. That's About to Change by andai on Wednesday December 24, 2025 @05:13AM (#46374228)
Thanks. I call this method Power Coding (like Power Armor), where you're still doing everything except for typing out the syntax.

I found that for this method the smaller the model, the better it works, because smaller models can generally handle it, and you benefit more from iteration speed than anything else.

I don't have hardware to run even tiny LLMs at anything approaching interactive speeds, so I use APIs. The one I ended up with was Grok 4 Fast, because it's weirdly fast.

ArtificialAnalysis has a section "end to end" time, and it was the best there for a long time, tho many other models are catching up now.

Re:Laptop Isn't Ready for LLMs. That's About to Change by jwr on Wednesday December 24, 2025 @04:22AM (#46373960)
The speed is fine, the models are not.

I found only one great application of local LLMs: spam filtering. I wrote a "despammer" tool that accesses my mail server using IMAP, reads new messages, and uses an LLM to determine if they are spam or not. 95.6% correct classification rate on my (very difficult) test corpus, in practical usage it's nearly perfect. gpt-oss-20b is currently the best model for this.

For all other purposes models with <80B parameters are just too stupid to do anything useful for me. I write in Clojure and there is no boilerplate: the code reflects real business problems, so I need an LLM that is capable of understanding things. Claude Code, especially with Opus, does pretty well on simpler problems, all local models are just plain dumb and a waste of time compared to that, so I don't see the appeal yet.

That said, my next laptop will be a MacBook pro with M5 Max and 128GB of RAM, because the small LLMs are slowly getting better.

Re:Laptop Isn't Ready for LLMs. That's About to Change by layer8 on Tuesday December 23, 2025 @04:05PM (#46369445)
By “PC”, they mean non-Apple devices.

Also, macOS only has around 10% desktop market share globally.

Re:Laptop Isn't Ready for LLMs. That's About to Change by dangus on Tuesday December 23, 2025 @10:52PM (#46372253)
It's actually closer to 20% globally. Apple now outsells Lenovo:

https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-t...

Re:Laptop Isn't Ready for LLMs. That's About to Change by layer8 on Wednesday December 24, 2025 @08:55AM (#46375590)
I meant market share in terms of installed base: https://gs.statcounter.com/os-market-share/desktop/worldwide...
Re:Laptop Isn't Ready for LLMs. That's About to Change by dangus on Wednesday December 24, 2025 @10:04AM (#46376176)
macOS and OS X are split on this graph, and “Unknown” could be anything? This might actually show Apple install base close to 20%.
Re:Laptop Isn't Ready for LLMs. That's About to Change by fancyfredbot on Tuesday December 23, 2025 @04:26PM (#46369653)
Most laptops have 16GB of RAM or less. A little more than a year ago I think the base model Mac laptop had 8GB of RAM which really isn't fantastic for running LLMs.
Re:Laptop Isn't Ready for LLMs. That's About to Change by literalAardvark on Tuesday December 23, 2025 @04:26PM (#46369654)
Apple has a 10-18% market share for laptops. That's significant but it certainly isn't "most".

Most laptops can run at best a 7-14b model, even if you buy one with a high spec graphics chip. These are not useful models unless you're writing spam.

Most desktops have a decent amount of system memory but that can't be used for running LLMs at a useful speed, especially since the stuff you could run in 32-64GB RAM would need lots of interaction and hand holding.

And that's for the easy part, inference. Training is much more expensive.

Re:Laptop Isn't Ready for LLMs. That's About to Change by seanmcdirmid on Tuesday December 23, 2025 @06:53PM (#46370910)
A Max cpu can run 30b models quantized, and definitely has the RAM to fit them in memory. The normal and pro CPUs will be compute/bandwidth limited. Of course, the Ultra CPU is even better than the Max, but they don't come in laptops yet.
Re:Laptop Isn't Ready for LLMs. That's About to Change by nunodonato on Tuesday December 23, 2025 @06:08PM (#46370558)
my laptop is 4 years old. I only have 6Gb VRam. I run, mostly, 4b and 8b models. They are extremely useful in a variety of situations. Just because you can't replicate what you do in chatgpt doesn't mean they don't have their use cases. It seems to me you know very little about what these models can do. Not to speak of trained models for specific use cases, or even smaller models like functiongemma or TTS/ASR models. (btw, I've trained models using my 6Gb VRAM too)
Re:Laptop Isn't Ready for LLMs. That's About to Change by literalAardvark on Tuesday December 23, 2025 @08:25PM (#46371455)
A few examples of useful tasks would be appreciated. I do suffer from a sad lack of imagination.
Re:Laptop Isn't Ready for LLMs. That's About to Change by nunodonato on Tuesday December 23, 2025 @08:30PM (#46371487)
I suggest taking a look at /r/localLLaMa and see all sorts of cool things people do with small models.
Re:Laptop Isn't Ready for LLMs. That's About to Change by reactordev on Tuesday December 23, 2025 @08:32PM (#46371497)
I’ll chime in and say I run LM Studio on my 2021 MacBook Pro M1 with no issues.

I have 16GB ram. I use unsloth quantized models like qwen3 and gpt-oss. I have some MCP servers like Context7 and Fetch that make sure the models have up to date information. I use continue.dev in VSCode or OpenCode Agent with LM Studio and write C++ code against Vulkan.

It’s more than capable. Is it fast? Not necessarily. Does it get stuck? Sometimes. Does it keep getting better? With every model release on huggingface.

Total monthly cost: $0

Re:Laptop Isn't Ready for LLMs. That's About to Change by DANmode on Tuesday December 23, 2025 @05:05PM (#46370028)
> Apple has a huge marketshare in the laptop market

Hello, from outside of California!

Re:Laptop Isn't Ready for LLMs. That's About to Change by dangus on Tuesday December 23, 2025 @10:53PM (#46372255)
Global Mac marketshare is actually higher than the US: https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-t...
Re:Laptop Isn't Ready for LLMs. That's About to Change by DANmode on Tuesday December 23, 2025 @11:03PM (#46372313)
Less than 1 in 5 doesn’t feel like huge market share,

but it’s more than I have!

Re:Laptop Isn't Ready for LLMs. That's About to Change by robotresearcher on Wednesday December 24, 2025 @12:06AM (#46372650)
It includes the single best selling laptop model from any maker: the Macbook Air.

I don’t know, but would not be surprised if Apple’s share of global laptop revenues was significantly larger than their share of units sold. I believe that’s true of smartphones.

Re:Laptop Isn't Ready for LLMs. That's About to Change by dangus on Wednesday December 24, 2025 @10:02AM (#46376169)
Apple outsells Lenovo, if that puts it in a different perspective.