News for hackers. Stuff that matters.
Re:AMD acquires Taalas to boost inference performance by etc... by andix on Thursday August 06, 2026 @08:20PM (#49204438)
With thousands of token per second output it would be an enormous waste of resources. Such chips are clearly made to process thousands of conversations simultaneously. Not necessarily in parallel. All LLM workflows are turn based right now, there are often seconds between turns until tool calls finish or users type the next message.

If the LLM response only takes a few milliseconds, the chip can process hundreds of other requests until the first conversation becomes active again.

Re:AMD acquires Taalas to boost inference performance by etc... by akie on Friday August 07, 2026 @04:24AM (#49207411)
> it would be an enormous waste of resources

Sounds a lot like "640Kb ought to be enough for anybody"

Re:AMD acquires Taalas to boost inference performance by etc... by tiborsaas on Friday August 07, 2026 @06:49AM (#49208436)
With those speeds I can benchmark a batch of different approaches, compare the results and serve the results all within a second. It's quite amazing.