News for hackers. Stuff that matters.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by freehorse on Wednesday October 16, 2024 @09:48PM (#41865681)
I always thought that the main point of NPUs is energy efficiency (and being able to run ML models without taking over all computer resources, making it practical to integrate ML applications in the OS itself in ways that it does not disturb the user or the workflow) rather than being exceptionally faster. At least this has been my experience with running stable diffusion on macs. Similar to using other specialised hardware like media encoders; they are not necessarily faster than a CPU if you throw a dozen+ cpu cores on the task, but it will draw a minuscule part of the power.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by tokyolights on Sunday October 20, 2024 @09:20PM (#41899887)
Not sure why this isn't discussed more here. I think exactly the same, the NPU occupies more silicon area because it has custom circuits specifically to reduce the number of cycles (and thus energy) needed to perform those calculations. Doesn't necessarily mean that a CPU wouldn't be able to bulldoze through it faster (with much more energy).