News for hackers. Stuff that matters.
Re:AMD acquires Taalas to boost inference performance by etc... by wolttam on Thursday August 06, 2026 @05:57PM (#49203171)
It's a terrible moat. You etch the silicon then nobody wants to run it in 6 months because models have advanced that much further.
Re:AMD acquires Taalas to boost inference performance by etc... by nine_k on Thursday August 06, 2026 @06:05PM (#49203250)
Not so if it's embedded in something smart enough for its intended purpose.

Think vision, spatial reasoning, speech synthesis, even some speech analysis. Think self-driving cars (and drones) that need 10x less power for the brain, and can think at 10x situation per second.

Re:AMD acquires Taalas to boost inference performance by etc... by anigbrowl on Thursday August 06, 2026 @06:30PM (#49203521)
This is only true for people who are solely focused on performance. There is absolutely a market for acceptable performance combined with predictability.
Re:AMD acquires Taalas to boost inference performance by etc... by teraflop on Thursday August 06, 2026 @07:42PM (#49204167)
True, but predictability cuts both ways.

We're all used to having to constantly update our browsers and phones to keep up with the security arms race. If a frozen model can't be updated, it will predictably remain vulnerable to any "exploits" or idiosyncratic quirks that people discover over time.

Let's say, as somebody suggested in another comment, that you buy 100,000 of these chips and deploy them to run fast-food drive-thrus. And then somebody discovers the model has a fondness for goblins[1], and if you role-play convincingly enough, you can get it to accept payment in shiny buttons and rodent skulls instead of cash.

What do you do then? I guess your options are to try and fix the behavior with a better prompt, or put some kind of filter in front of the model to catch attempted exploits. If the filter is cheap and dumb it probably won't work well enough, and if you use another model as a filter, you've negated the cost and speed benefits of putting the first model in hardware.

Of course the real answer is to just never expose the model to situations where an adversarial input could possibly lead to an undesired output. But that drastically limits what you can do with it.

[1]: https://openai.com/index/where-the-goblins-came-from/

Re:AMD acquires Taalas to boost inference performance by etc... by noisy_boy on Thursday August 06, 2026 @09:53PM (#49205046)
> Of course the real answer is to just never expose the model to situations where an adversarial input could possibly lead to an undesired output. But that drastically limits what you can do with it.

Does it though? Isn't that what CPUs are, very fast-not-so-clever computing brain surrounded by layers that protect it?

Re:AMD acquires Taalas to boost inference performance by etc... by anigbrowl on Thursday August 06, 2026 @10:08PM (#49205159)
I see your argument but your example seems highly contrived. I can't think why you'd want to use something like this for something as dynamic as takeout ordering, where you might have to deal with bad customers, supply chain breakages, public health recalls, or any of many other probabilistic events.

I think it's far more likely to see them used in safety critical applications where you need a capable model that can run on low power and doesn't have multiple layers of operating abstractions between the model and the hardware.

Re:AMD acquires Taalas to boost inference performance by etc... by fwip on Thursday August 06, 2026 @11:01PM (#49205468)
What safety critical applications would be a good fit for LLMs?
Re:AMD acquires Taalas to boost inference performance by etc... by anigbrowl on Friday August 07, 2026 @12:56PM (#49213261)
I'm not thinking of language models specifically, but large neural networks in silico. I feel like a 27B parameter model would likely be capable of flying and landing an airliner, for example.
Re:AMD acquires Taalas to boost inference performance by etc... by twobitshifter on Thursday August 06, 2026 @07:01PM (#49203811)
OTOH, people get a new iPhone every year and they are ok with it.
Re:AMD acquires Taalas to boost inference performance by etc... by nomel on Thursday August 06, 2026 @07:09PM (#49203876)
How is that in any way related to a consumer device? This method doesn't reduce physical memory requirements, so still results in huge die area. This isn't a for-end-user thing, probably for decades.
Re:AMD acquires Taalas to boost inference performance by etc... by twobitshifter on Thursday August 06, 2026 @07:11PM (#49203891)
Ok, how long until nvidia gives us a new GPU?
Re:AMD acquires Taalas to boost inference performance by etc... by nomel on Thursday August 06, 2026 @07:17PM (#49203939)
I don't follow. How is that related? GPUs don't have fixed memory. You don't throw them away when you want to load a new model.

NVIDIA will probably give us a new GPU when someone competent in the free market decides they want wheelbarrows full of money. Unfortunately, AMD is entirely, incomprehensibly, incompetent, to the point where I can only assume they're colluding with Nvidia, behind the scenes.

Re:AMD acquires Taalas to boost inference performance by etc... by twobitshifter on Saturday August 08, 2026 @11:13AM (#49222605)
Point being that hardware generations can be very quick and as updates get made GPUs go out of date and can’t run the latest models. All types of hardware consumer of otherwise are always improving, which means that tying a model the hardware is not going to lead to increased obsolescence, any more than the hardware itself does. If an AI is general purpose then there is no problem with only having that one model baked into the chip, when the benefit is a huge speed and efficiency improvement over running it on a GPU.
Re:AMD acquires Taalas to boost inference performance by etc... by nomel on Monday August 10, 2026 @05:05PM (#49249729)
I think there's some confusion about what's going on here.

A GPU is general purpose, for inference sake. You can run any model that can fit in it. It will be obsolete, as all hardware eventually is, but a 3090 today is more useful than a 3090 two years ago, because small models have improved significantly.

Hardware as a model can run exactly one model, ever. You can't try a fine tune, and can't try the new similarly sized model that's better than all then others you've ever tried. You can run exactly one set of weights, with the architecture it shipped with, because everything is fixed.

Re:AMD acquires Taalas to boost inference performance by etc... by speed_spread on Thursday August 06, 2026 @06:03PM (#49203231)
If a model is good enough today, it's still gonna be good enough in a year. Except you'll be able to serve it 1/100 of the price. Or 100x the speed.
Re:AMD acquires Taalas to boost inference performance by etc... by wolttam on Friday August 07, 2026 @11:24AM (#49211920)
I think we will eventually reach a point where this is the case, but at the moment it seems like you can throw virtually any non-trivial use-case at a model today and end up being more satisfied with the results that a model tomorrow gives.

I may just be closed minded as to what use-cases we have that current models are truly "good enough" (i.e. won't be dissatisfied when comparing results of today's model to tomorrow's model)

Re:AMD acquires Taalas to boost inference performance by etc... by askl on Friday August 07, 2026 @06:55AM (#49208488)
Might be worse because it takes time between designing the silicon and having the first usable chips. So they're outdated the moment they hit the market or even before that.