News for hackers. Stuff that matters.
Re:AMD acquires Taalas to boost inference performance by etc... by RussianCow on Thursday August 06, 2026 @06:43PM (#49203651)
That likely isn't as relevant for on-device iPhone usage as it is for Real Work™. I won't notice the difference between 50tps and 1000tps when asking Siri a question.
Re:AMD acquires Taalas to boost inference performance by etc... by spijdar on Thursday August 06, 2026 @07:11PM (#49203898)
I don't know. As others have said, the Taalas chip wasn't small, or particularly low power, so it's hard to "imagine" what that tech in an cell phone chip might look like.

But if the basic premise of "good enough LLM at insane throughput" holds, I think it could qualitatively change local uses of LLMs. At a certain speed point, you're able to move from request -> response to a cascade of tool calling and "subagents", which could allow a small model to be much more useful, if provided with a lot of local data and tool calls.

That said, this is assuming you could stuff a "good enough" model into a phone with Taalas-like technology. The Taalas tech demo was an 8B parameter model and required hundreds of watts (IIRC) to run. The efficiency was good given the speed (as I understand), but it's not clear at all that the approach scales small enough to be a sensible coprocessor on an iPhone or whatever.

Re:AMD acquires Taalas to boost inference performance by etc... by intrasight on Thursday August 06, 2026 @08:23PM (#49204465)
Box that plugs into my desktop would be fine. Or perhaps in SSF form factor.
Re:AMD acquires Taalas to boost inference performance by etc... by manmal on Friday August 07, 2026 @01:45AM (#49206295)
> At a certain speed point, you're able to move from request -> response to a cascade of tool calling and "subagents"

That also needs server class hardware though. A phone won’t happily service the insane amount of IO, compute, and network that this cascade would require.

Re:AMD acquires Taalas to boost inference performance by etc... by retatop on Thursday August 06, 2026 @07:20PM (#49203964)
But wouldn't higher tps allow for more reasoning or other hidden processes, potententially making a smarter model?
Re:AMD acquires Taalas to boost inference performance by etc... by dabbz on Thursday August 06, 2026 @07:58PM (#49204300)
This is my thought as well. Models have to be intentional about which tokens they burn because there's a real lag time. If you can just fork out 10 different reasoning sessions at once with no regard for token waste/lag, you can compensate a smaller model with just doing more at once with it. No idea if this is reasonably true though.
Re:AMD acquires Taalas to boost inference performance by etc... by manmal on Friday August 07, 2026 @01:35AM (#49206240)
I think that only works if you have checkpoints where all that reasoning can be checked against reality. Otherwise you get an army of armchair experts. LLMs are hilariously bad at home improvement advice btw, where reasoning alone won’t get you far.
Re:AMD acquires Taalas to boost inference performance by etc... by nvme0n1p1 on Thursday August 06, 2026 @07:22PM (#49203973)
That order of magnitude could be the difference between "the users wants me to open the notes app, let's open it" and "I've scanned all your notes before you could blink and found what you're looking for".
Re:AMD acquires Taalas to boost inference performance by etc... by p1esk on Thursday August 06, 2026 @07:39PM (#49204141)
If Siri is using a 3T model in high reasoning mode to answer your question you will.