AI PCs Aren't Good at AI: The CPU Beats the NPU

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by AlexAndScripts on Wednesday October 16, 2024 @06:47PM (#41864643)

It almost makes me want to find some use for them on my Linux box (not that is has an NPU), but I truly can't think of anything. Too small to run a meaningful LLM, and I'd want that in bursts anyway, I hate voice controls (at least with the current tech), and Recall sounds thoroughly useless. Could you do mediocre machine translation on it, perhaps? Local github copilot? An LLM that is purely used to build an abstract index of my notes in the background?

Actually, could they be used to make better AI in games? That'd be neat. A shooter character with some kind of organic tactics, or a Civilisation/Stellaris AI that doesn't suck.

[ Parent | on HN ]

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by ywvcbk on Thursday October 17, 2024 @04:42AM (#41867610)

> box

Presumably you have a GPU? If so there is nothing an NPU can do that a discrete GPU can’t (and it would be much slower than a recent GPU).

The real benefits are power efficiency and cost since they are built into the SoC which are not necessarily that useful on a desktop PC.

[ Parent | on HN ]

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by Miraste on Thursday October 17, 2024 @02:16PM (#41872169)

In short: no. Current-gen NPUs are so slow they can't do anything useful. AMD and Intel have 2nd-gen ones that came out a few weeks ago, and by spec they may able to run local translation and small LLMs (haven't seen benchmarks yet), but for now they are laptop-only.

[ Parent | on HN ]