News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by ThePinion on Wednesday November 11, 2026 @02:02AM (#50450486)
I have the same Mac M4 16gb. Do you have the Qwen model assist the larger models for menial tasks/tool calls during coding it can handle? As in have them work in tandem to save on tokens/calls on the large models or are your local agents just meant for their own specific side tasks?
Re:Apple has new Mac app coming to make running local AI mod... by brainless on Wednesday November 11, 2026 @02:10AM (#50450536)
A mix of both. I have been experimenting building agents that focus on small/tiny LLMs. For example, I built a web-app prototyping harness which used Qwen 4/9B models but separate agents (tool calls, prompts and actual code to parse) for CSS management, TSX management, Router, state, etc. It used SolidJS only.

There was a task manager which was given context of the tech stack so it could split the incoming requests. Then branch out the tasks to appropriate agents. Combine the generated code - this was a file/module/import manager.

The other way is let tiny models (again I would use agents for different use cases) do the busy work - tool calling, finding evidence, etc. I usually setup agents to re-write tool call history and create fresh API requests without using multi-turn chat with tool call history.

Re:Apple has new Mac app coming to make running local AI mod... by ThePinion on Wednesday November 11, 2026 @02:56PM (#50457812)
I really appreciate the response, that gives me good ideas on where to start going about all this. It seems so silly to have API models doing so much file exploration and tool calling.

I saw this a while ago which shows small models and focuses on the ones that make the least (or no) wrong tool calls and it got me intrigued on the possibilities in this realm, especially for lower end devices. https://github.com/MikeVeerman/tool-calling-benchmark