Re:Apple has new Mac app coming to make running local AI mod...
by yieldcrv on Wednesday November 11, 2026 @04:11AM
(#50451213)
if you're running qwen, check out what happens when you swap oMLX for kapitsa
its built just for qwen, healthy competition on tokens/sec
Re:Apple has new Mac app coming to make running local AI mod...
by Maxion on Wednesday November 11, 2026 @05:36AM
(#50451832)
What is the experience of these local LLM models on this type of hardware? what are they capable of ? what do you use them for?
Re:Apple has new Mac app coming to make running local AI mod...
by pentakkusu on Wednesday November 11, 2026 @06:53AM
(#50452331)
A-OK, I would say. The pinnacle would be that I managed to develop a purpose made android app for my 2016 Honda dash unit with Hermes.
Hermes runs my personal to-do, manages 2 calendars, triages forwarded bills via it's own email address, orchestrates my docker container fleet. It also tends to a partly botched llm-wiki, that is built on Obsidian.
Re:Apple has new Mac app coming to make running local AI mod...
by yieldcrv on Wednesday November 11, 2026 @01:33PM
(#50456799)
I do agentic coding and agentic search and document analysis just like what I do with Claude
they have 128k - 256k context windows now, subagents use the same weights loaded in memory so you have spawn many subagents at the same time just like Claude Code, and the tokens/sec has is like 150-200
I just do different projects based on its complexity or liability or its moat
Re:Apple has new Mac app coming to make running local AI mod...
by pentakkusu on Wednesday November 11, 2026 @06:49AM
(#50452303)
never heard of it, will check!
I use oMLX btw, on 32GB URAM, running Hermes with Qwen 3.6 @ 128k context window.
I realize I am a smalltimer but for general day to day sensitive agentic stuff, this is the entry point.