Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by quotemstr on Monday July 20, 2026 @01:15PM
(#48981739)
Ah, right. I misremembered. Does Taalas have a meaningful advantage here then?
[ Parent
| on HN ]
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by lsaferite on Monday July 20, 2026 @01:45PM
(#48982179)
Taalas claims significant cost, speed, and power differential with running on Nvidia (and similar) hardware. Their whole thesis is that with making the hardware cost less, run faster, and run with less power mixed with their claimed fast production cycle for new weights, companies would plan on purchasing and then relegating the other chips to different workloads at the next update cycle. And the chips have facilities for fine-tunes that can by dynamically loaded. The info I've seen has been very light on details, but if they can even get halfway where they are planning, it'll be a massive shift.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by wyre on Monday July 20, 2026 @01:31PM
(#48982005)
6x. Taalas has Llama3.1 8B running at 18000 tok/s. Cerebras advertised that model at 3000 tok/s.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by quotemstr on Monday July 20, 2026 @01:35PM
(#48982053)
Interesting. Is the speedup from specializing for the shape of Llama 3.1 or are they (contra my mental model) actually winning on burning in the weights?
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by akiselev on Monday July 20, 2026 @01:59PM
(#48982397)
The weights are in SRAM, so the LLM architecture is burned in but the weights can be updated.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by hedgehog on Monday July 20, 2026 @03:10PM
(#48983485)
No the weights are in the metal layers, they cannot be updated.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by jfim on Monday July 20, 2026 @07:15PM
(#48986092)
The base weights can't be updated but from what I recall it allows adding a low rank adapter to customize the model a little bit.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by hedgehog on Tuesday July 21, 2026 @10:42AM
(#48992971)
Yes that's what I've read. As far as I know the approach should transfer well to hybrid model architectures like modern Qwen and sizes like 27B by using multiple chips. LoRA-steered Qwen 27B at 10K+ tokens per second would be transformative for some workflows.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by Tuna-Fish on Monday July 20, 2026 @05:55PM
(#48985376)
Taalas only uses SRAM for the KV cache and the activations, the weights are in mask rom in the metal layers.
If they designed this right, it means that once they have a model, so long as they keep the hyperparameters fixed they can change the weights much faster than it takes to spin up a completely new chip, essentially at a cost of doing a minor revision.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by quotemstr on Monday July 20, 2026 @07:04PM
(#48985981)
Is the mask ROM really going to be worth it over Carmack's high-bandwidth-flash concept? I mean, sure, I could be convinced I guess, but it's not obvious.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by Tuna-Fish on Tuesday July 21, 2026 @04:42PM
(#48998002)
Yes.
Not because rom is better than flash, but because the critical part is distributing the memory with the compute. Mask ROM is just the densest way of embedding memory with logic. Instead of having a large pool of memory connected to the separate execution units with a bus, each execution unit locally has the rom that it uses. Data movement is >90% of energy use in modern ai accelerators, removing it as far as practical is how they get performance and silicon and energy efficiency.
Re:Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling
by tacitusarc on Tuesday July 21, 2026 @10:02PM
(#49000937)
But attention/KV cache is the heavy bit in long context lengths which is what everyone needs…