News for hackers. Stuff that matters.
Re:Ollama is now powered by MLX on Apple Silicon in preview by testing22321 on Tuesday March 31, 2026 @10:44AM (#47588093)
I see all these LLM posts about if a certain model can run locally on certain hardware and I don’t get it.

What are you doing with these local models that run at x tokens/sec.

Do you have the equivalent of ChatGPT running entirely locally? What do you do with it? Why? I honestly don’t understand the point or use case.

Re:Ollama is now powered by MLX on Apple Silicon in preview by svachalek on Tuesday March 31, 2026 @12:17PM (#47589666)
1. There are small local models that have the capabilities of frontier models a year ago

2. They aren't harvesting your data for government files or training purposes

3. They won't be altered overnight to push advertising or a political agenda

4. They won't have their pricing raised at will

5. They won't disappear as soon as their host wants you to switch

Re:Ollama is now powered by MLX on Apple Silicon in preview by testing22321 on Tuesday March 31, 2026 @04:47PM (#47593282)
Thanks. I understand that.

What are you doing with it?

Why do you want it?

Re:Ollama is now powered by MLX on Apple Silicon in preview by jkl5xx on Tuesday March 31, 2026 @05:38PM (#47593805)
Good points. What local models have you found work best for your use cases? I feel like if we get to opus 4.6 level intelligence running on local hardware, we’re in the clear for a lot of day to day use cases.
Re:Ollama is now powered by MLX on Apple Silicon in preview by samuel on Tuesday March 31, 2026 @10:50AM (#47588199)
Chat is certainly an option, but the real deal are agents, which have access to way more sensitive information.
Re:Ollama is now powered by MLX on Apple Silicon in preview by testing22321 on Tuesday March 31, 2026 @04:48PM (#47593287)
Thanks. What do you do with such an agent? What is the use case?
Re:Ollama is now powered by MLX on Apple Silicon in preview by dec0dedab0de on Tuesday March 31, 2026 @11:18AM (#47588620)
most of the llm tooling can handle different models. Ollama makes it easy to install and run different models locally. So you can configure aider or vscode or whatever you're using to connect to chatgpt to point to your local models instead.

None of them are as good as the big hosted models, but you might be surprised at how capable they are. I like running things locally when I can, and I also like not worrying about accidentally burning through tokens.

I think the future is multiple locally run models that call out to hosted models when necessary. I can imagine every device coming with a base model and using loras to learn about the users needs. With companies and maybe even households having their own shared models that do heavier lifting. while companies like openai and anhtropic continue to host the most powerful and expensive options.

Re:Ollama is now powered by MLX on Apple Silicon in preview by roboror on Tuesday March 31, 2026 @02:13PM (#47591338)
What models have you found capable? I was recently recommended Qwen3 Coder Next and I did not find it very successful. I have a good amount of VRAM/RAM so would love to run something locally.
Re:Ollama is now powered by MLX on Apple Silicon in preview by testing22321 on Tuesday March 31, 2026 @05:36PM (#47593779)
Thanks.

I still don’t understand. What are you using this long you’re running locally to actually do?

What is the use case?

Re:Ollama is now powered by MLX on Apple Silicon in preview by derangedHorse on Wednesday April 01, 2026 @07:16AM (#47599357)
Qwen3.5 is like an old version of ChatGPT and I can use it the same way I used GPT4 — writing emails, reading documentation and answering questions about it, reviewing code, answering trivia, etc.