Accelerating GPT-5.6 Sol Ultrafast

Re:Accelerating GPT-5.6 Sol Ultrafast by JohnBooty on Thursday August 13, 2026 @11:51PM (#49294559)

    Shall the better model still have 
    the upper hand or will the raw speed 
    compensate?

At 14,000 tokens/sec there's just so much ridiculous stuff that might be possible. Let's assume that this POC proves they can take the next step, and can eventually etch a capable ~27B model into silicon. Let's call it Fred.

Ralph loops automatically get real real interesting again. 200x the iteration speed. This is such a clear win I feel like there's hardly anything to talk about. Instead of one stubborn iterating idiot, you could have dozens of idiots competing in parallel, genetic algorithm style.

The other common orchestration pattern I see is "big model for planning, small parallel subagents implementing, big model reviewing" Today it's Sol dispatching a handful of Luna subagents. Tomorrow maybe it's Sol dispatching as many Fred subagents as it could possibly want.

But what patterns have we not even thought about yet in a world where subagents are 200x faster/cheaper?

What if instead of dispatching single Haiku/Luna/etc subagents, we dispatched "teams" of Fred agents? Maybe each team is 8 Freds. Five come up with competing ideas and the other three vote on a winner.

Or what if they were heterogenous teams? One Luna and a bunch of Freds.

What if instead of a two-tier orchestration system (Sol->Luna) it was three-tier or n-tier? (Sol->Luna->Fred->...Fred)

Those ideas overlap a bit, and crazy shit like Gas Town has already explored even wilder ideas I guess. But man, 14000 tk/sec opens up so much stuff.

[ Parent | on HN ]

Re:Accelerating GPT-5.6 Sol Ultrafast by hadlock on Friday August 14, 2026 @01:28AM (#49295027)

At 14,000t/s that's effectively a motor cortex for an android, you no longer need to train the robot to walk, it has a general idea for how to walk (baked into the 1b model), and then just corrects based on sensor input, in real time.

[ Parent | on HN ]

Re:Accelerating GPT-5.6 Sol Ultrafast by fennecfoxy on Friday August 14, 2026 @09:22AM (#49298348)

I still personally think that a heavy lean into MoE will be better for that sort of thing. Our brains are subdivided into large parts but I'm sure (and I'm not a brain scientist) that those parts can be subdivided even further into systems that run at various frequencies and latencies depending on what they're used for.

I was thinking about it the other day actually. How our brains evolved structure. I imagine it was purely just down to evolution adding/clustering additional cells around the areas where additional cells were needed. And after long enough a natural brain architecture emerged.

Makes me wonder if we're on the right track with transformer architecture/attention but if it'd be more effective on a larger scale, like MoE with a billion "experts".

[ Parent | on HN ]

Re:Accelerating GPT-5.6 Sol Ultrafast by JohnBooty on Friday August 14, 2026 @07:32PM (#49305868)

    like MoE with a billion "experts".

That seems promising to me too, although, the thing I've always read is that you can't make the "experts" too narrow. Even if you had a "coding expert" it has to know a lot more than coding - if you tell it to make an online store it needs to parse your language, understand the internet, what a "store" is in this context, etc.

I am not a primary source, probably not even a secondary or tertiary source, so take this with all the grains of salt.

[ Parent | on HN ]

Re:Accelerating GPT-5.6 Sol Ultrafast by fennecfoxy on Wednesday August 19, 2026 @07:13AM (#49359940)

Oh for sure, but I think generally multiple experts are selected in an MoE pass for a token, so presumably it'd select programming related ones as well as general knowledge/language.

Only problem would be the routing layer works on the previous token as far as I understand so it might need more informational depth than just "a token" to select experts, I suppose in the same way attention works.

I wish I had the GPUs to run those sorts of experiments ha ha.

[ Parent | on HN ]

Re:Accelerating GPT-5.6 Sol Ultrafast by sixtyj on Friday August 14, 2026 @03:04AM (#49295521)

Yes, imho 14k TPS is just a beginning.

This Gas Town? https://github.com/gastownhall/gastown

[ Parent | on HN ]