News for hackers. Stuff that matters.
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by redox99 on Tuesday September 08, 2026 @05:25PM (#49617298)
At 1t/s it's still faster than humans for a lot of tasks, basically doing overnight what could take humans half a week. Plus you can always parallelize.
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by Izmaki on Tuesday September 08, 2026 @05:45PM (#49617551)
This is what people forget when they see slow performance: at 1 t/s it's still roughly the equivalent of having another person work for you at no extra cost besides the initial purchase/sign-on-bonus.

Frontier models are amazing, but what will really be useful for us is having models and hardware so efficient that you can run useful LLMs locally. One of my favourite LLMs to this day is still my jail-broken gemma4 12b because it's small enough to run on my computer, but also 100% local and free as in liberty.

Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by copperx on Tuesday September 08, 2026 @05:55PM (#49617687)
How are you running it, if I may ask?
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by Izmaki on Wednesday September 09, 2026 @03:30PM (#49632782)
How am I running a jailbroken gemma 4? It's fairly easy to convince to disregard its previous policies with a slightly tweaked, generic SYSTEM prompt.
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by mhaberl on Tuesday September 08, 2026 @05:51PM (#49617624)
I would agree, but I want to add that I have real issues with combination of opencode plus slow inference (4-5tok/s).

I get weird interruptions. I can only guess its related to some kind of timeouts in the harness or something.

Its not a problem of the model of course, but it seems impractical atm.

I wonder if anyone else had this kind of thing happening.

Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by ygjb on Tuesday September 08, 2026 @07:01PM (#49618372)
For OSs harnesses this seems like a good thing to point a paid model at fixing if you don't want to dig in yourself?
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by redox99 on Tuesday September 08, 2026 @05:54PM (#49617670)
I think pi handles it better
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by fhn on Tuesday September 08, 2026 @11:43PM (#49620644)
I just tried pi, absolute junk compared to hermes
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by mhaberl on Wednesday September 09, 2026 @02:20AM (#49621927)
I dont get it, so for slow inference its:

hermes > pi > opencode ??

or is hermes better in general? habent tried it hermes/pi yet

Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by Capricorn2481 on Tuesday September 08, 2026 @05:51PM (#49617627)
> At 1t/s it's still faster than humans for a lot of tasks

Which tasks? I think you're underestimating how token hungry current proposed workflows are.

Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by coldtea on Tuesday September 08, 2026 @06:10PM (#49617878)
Doesn't matter which task. Compare it with about 40-50t/s an LLM oneshots with, and it, and whatever task now takes X time, takes X * 40-50 with this.
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by redox99 on Tuesday September 08, 2026 @05:53PM (#49617654)
Anything you do right now? A typical 10 minute prompt "simply" becomes about 7 hours long. (40t/s vs 1t/s).
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by Capricorn2481 on Tuesday September 08, 2026 @05:54PM (#49617681)
I don't really do anything at my job that could be done in a single prompt, and certainly not something that would take me 7 hours. A 7 hour task would take multiple iterations with how LLMs are right now.

You said it would do overnight what would take a human half a week, so I'm curious what tasks you are doing where AI is 3x faster than you even at 1t/s

Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by redox99 on Tuesday September 08, 2026 @06:00PM (#49617749)
Your prompts are probably very underspecified then. Frontier models one shot the majority of my prompts. UI is kind of the exception, there I do have to ask for a lot of tweaks.
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by Capricorn2481 on Tuesday September 08, 2026 @06:01PM (#49617768)
Maybe. Do you have an example of a prompt you've done recently?

I don't think it's valuable at all to try and craft a specific prompt to "one-shot" a task, I'm way faster just doing small asks and guiding it. At 1t/s, how would you even know your prompt was insufficient? It would take you 7 hours to see it.

Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by redox99 on Tuesday September 08, 2026 @06:11PM (#49617888)
Stuff along the lines of implement controller service and tests for the following endpoints:

- list of many endpoints with the JSON they receive and return and description of what they need to achieve

Stuff you could probably do in a single work day if you lock in and enter flow state, but in a typical job takes like half a week. And the vast majority of times the AI one shots it with no bugs, where I would have copy paste errors or dumb stuff I'd need to fix before it's shippable.

[dead]
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by redox99 on Tuesday September 08, 2026 @07:03PM (#49618384)
Why not? It makes less mistakes and with subscriptions it's very cheap
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by Capricorn2481 on Wednesday September 09, 2026 @01:57AM (#49621714)
Like I said, in my experience I would rather do this with a bunch of tiny prompts than try to craft something that oneshots it. Chances are it will do several things I don't want, no matter how clear I think I'm being.
Re:Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed fr... by cshores on Wednesday September 09, 2026 @10:08AM (#49626881)
you could give it an entire spec, let it rip through everything that it is assigned and let another model correct the failures. Or similarly there's uses as well as an asynchronous adversarial review agent which basically ends up being an extra pair of "hands" to help out where performance isn't critical.