News for hackers. Stuff that matters.
Re:Laptop Isn't Ready for LLMs. That's About to Change by seanmcdirmid on Tuesday December 23, 2025 @06:57PM (#46370929)
Running an LLM locally means you never have to worry about how many tokens you've used, and also it allows for a lot of low latency interactions on smaller models that can run quickly.

I don't see why consumer hardware won't evolve to run more LLMs locally. It is a nice goal to strive for, which consumer hardware makers have been missing for a decade now. It is definitely achievable, especially if you just care about inference.

Re:Laptop Isn't Ready for LLMs. That's About to Change by KellyCriterion on Wednesday December 24, 2025 @05:00AM (#46374171)
isnt this what all these NPUs are created for?
Re:Laptop Isn't Ready for LLMs. That's About to Change by seanmcdirmid on Wednesday December 24, 2025 @12:16PM (#46377331)
I haven’t seen an NPU that can compete with a GPU yet. Maybe for really small models, I’m still not sure where they are going with those.