News for hackers. Stuff that matters.
Re:AMD acquires Taalas to boost inference performance by etc... by adrianN on Thursday August 06, 2026 @11:00PM (#49205462)
It is my understanding that just baking the model itself into silicon only gives moderate gains because memory bandwidth remains a bottleneck.
Re:AMD acquires Taalas to boost inference performance by etc... by chorizo on Thursday August 06, 2026 @11:07PM (#49205501)
The big benefit is ROM cells require fewer components than DRAM. So the chips would be tiny, dense, cheap and consume far less power.
Re:AMD acquires Taalas to boost inference performance by etc... by klodolph on Thursday August 06, 2026 @11:38PM (#49205670)
I thought DRAM was pretty dense already. Is mask ROM that much denser?
Re:AMD acquires Taalas to boost inference performance by etc... by chorizo on Thursday August 06, 2026 @11:50PM (#49205716)
Yes, each rom bit can be a transistor or even a diode with a decoder circuit. Simplest Dram cell is capacitor+transistor - and you need a clock, refresh circuit etc.

Someday, I imagine model weights could even be encoded as analog resistors (memristors or similar) for even greater density

Re:AMD acquires Taalas to boost inference performance by etc... by ReactiveJelly on Friday August 07, 2026 @12:41AM (#49205960)
Hm. I wonder how many relays I'd need to make a physical MNIST classifier. That'd be dope
Re:AMD acquires Taalas to boost inference performance by etc... by Tuna-Fish on Friday August 07, 2026 @08:58AM (#49209732)
It's not even just that. If you just built the rom chips separately and swapped them for the RAM of a normal accelerator, it would not help at all.

The trick is that every compute element in their system has it's own small pool of ROM, instead of putting all the ram behind a common pipe. ROM is just used because it's the densest kind of memory that can be fabricated on the same process as their logic.

Re:AMD acquires Taalas to boost inference performance by etc... by lelanthran on Friday August 07, 2026 @02:03AM (#49206422)
Did you try using the the talaas chat? Something stupid like 18k tokens/second.

Think it's called Askjimmy or similar.

Re:AMD acquires Taalas to boost inference performance by etc... by TeMPOraL on Friday August 07, 2026 @03:52AM (#49207203)
What is the model they're using there though? Interrogated, it claims it's a BERT variant and has capabilities around GPT-3 and below GPT-4.

(Not that I believe it, it writes too well for GPT-3.)

Hosted frontier models from two years ago would be much faster today, too.

Re:AMD acquires Taalas to boost inference performance by etc... by egorfine on Friday August 07, 2026 @04:18AM (#49207373)
They run Llama 3.1 8B.
Re:AMD acquires Taalas to boost inference performance by etc... by jgilias on Friday August 07, 2026 @02:16AM (#49206536)
Oh boy, thanks for sharing this, truly mind blowing. It was chatjimmy.ai