News for hackers. Stuff that matters.
Re:AMD acquires Taalas to boost inference performance by etc... by zxspectrum1982 on Thursday August 06, 2026 @06:45PM (#49203665)
I'd gladly pay for a Claude Opus 4.6 Thinking High in silicon and use it for 1-2 years. It's good enough for many coding tasks.
Re:AMD acquires Taalas to boost inference performance by etc... by andix on Thursday August 06, 2026 @08:20PM (#49204438)
With thousands of token per second output it would be an enormous waste of resources. Such chips are clearly made to process thousands of conversations simultaneously. Not necessarily in parallel. All LLM workflows are turn based right now, there are often seconds between turns until tool calls finish or users type the next message.

If the LLM response only takes a few milliseconds, the chip can process hundreds of other requests until the first conversation becomes active again.

Re:AMD acquires Taalas to boost inference performance by etc... by akie on Friday August 07, 2026 @04:24AM (#49207411)
> it would be an enormous waste of resources

Sounds a lot like "640Kb ought to be enough for anybody"

Re:AMD acquires Taalas to boost inference performance by etc... by tiborsaas on Friday August 07, 2026 @06:49AM (#49208436)
With those speeds I can benchmark a batch of different approaches, compare the results and serve the results all within a second. It's quite amazing.
Re:AMD acquires Taalas to boost inference performance by etc... by subroutine on Thursday August 06, 2026 @07:48PM (#49204219)
But Claude Opus 4.6 is not really practical. Taalas' process seems targeted for edge models. Their proof of concept model, for example, is a heavily quantized version of Llama 3.1 8B and even then they acknowledge their custom 3-bit/6-bit representation causes model quality degradation.

Taalas is going to have a tough time putting a trillion-parameter model on one conventional die. Their HC1 die is already near the maximum size that conventional lithography can expose. They claim they could partition the model across many chips, but I'm not sure if they have tested this process or what it means for compute. The basic storage arithmetic is unforgiving: for a one trillion parameters model at four bits it will take 50–100 chips. To service a sizable customer base will take thousands of 100-chip fabs.

That all said, I'm bullish on this technology, and look forward to seeing it evolve.

Re:AMD acquires Taalas to boost inference performance by etc... by vatsachak on Thursday August 06, 2026 @08:31PM (#49204514)
Yeah. But this kinda feels like a bandaid.

Eventually someone will have to solve compute in memory at scale.

Re:AMD acquires Taalas to boost inference performance by etc... by Iolaum on Friday August 07, 2026 @01:02AM (#49206061)
A really fast qwen-3.6-27B type of model could be useful. With a specialized harness and this speed I 'd expect it to find many applications. Implementing a coding plan is the minimum I can think of.
Re:AMD acquires Taalas to boost inference performance by etc... by momojo on Friday August 07, 2026 @11:57AM (#49212413)
I'm sure life would find a way. I'd love to see what kind of power-harnesses people have to come up with to steer 16k tps QPU's (Qwen Processing Units) productively.
Re:AMD acquires Taalas to boost inference performance by etc... by Gigachad on Thursday August 06, 2026 @07:09PM (#49203875)
It costs something like $300,000 for the hardware to run a model of that size. You'd pay that for a single model for 1-2 years? Not even the AI companies can justify that kind of spend which is why they keep extending the expected lifespan on their hardware in the accounting.
Re:AMD acquires Taalas to boost inference performance by etc... by zxspectrum1982 on Thursday August 06, 2026 @07:11PM (#49203892)
I'm expecting the Taalas MSIC version to cost a fraction of that. Then probably have some kind of cheap subscription to Anthropic for updates (yes, Taalas chips can receive a certain kind of updates: they have a small SRAM).
Re:AMD acquires Taalas to boost inference performance by etc... by mdp2021 on Thursday August 06, 2026 @08:03PM (#49204342)
> It costs something like $300,000 for the hardware to run a model of that size

You did not compute that as the cost for a speculative card from Taalas, right?

Re:AMD acquires Taalas to boost inference performance by etc... by Gigachad on Thursday August 06, 2026 @09:10PM (#49204739)
It's the cost of the current nvidia hardware used to run these models. Of course all bets are off if you are accounting for some future chip that doesn't exist yet which could cost less.
Re:AMD acquires Taalas to boost inference performance by etc... by lsaferite on Monday August 10, 2026 @03:10PM (#49248261)
Given the fact that Taalas was claiming 1/10th the hardware cost and 1/10th the power consumption, yeah, companies would absolutely jump at that. Now, whether they can achieve that in practice is yet to be seen.
Re:AMD acquires Taalas to boost inference performance by etc... by NiloCK on Thursday August 06, 2026 @10:55PM (#49205430)
Not so long ago, I was good enough for many coding tasks. But I found that things can change in a hurry.

Yes, a cheap and fast Opus4.6 can drive a lot of value in current context. But if we continue to craft bigger-and-bigger balls of mud, Opus 4.6 may end up hitting its conceptual ceiling and unable to contribute.

Winding the clock back on your statement gives:

> I'd gladly pay for a Claude Sonnet 3.5 in silicon and use it for 1-2 years.

Man, I dunno.

Re:AMD acquires Taalas to boost inference performance by etc... by kennywinker on Friday August 07, 2026 @12:04AM (#49205792)
Assuming moore's law like progress, which I'm 100% sure isn't going to happen - I think we're at the top of the S curve already. But assuming dramatically increased intelligence every year this is still the exact same position as anyone who bought a computer in the last 5 decades. Yet, people did very much buy computers.
Re:AMD acquires Taalas to boost inference performance by etc... by inigyou on Friday August 07, 2026 @09:11AM (#49209869)
But that's what people said 6 months ago about whichever model was current 6 months ago, but you hate that model now.