Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by fragmede on Thursday January 08, 2026 @04:45PM
(#46546944)
Do compilers know how to take advantage of that, or do programs need code that specifically takes advantage of that?
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by bfrog on Thursday January 08, 2026 @09:40PM
(#46549497)
It’s more like you need to program a dataflow rather than a program with instructions or vliw type processors. They still have operations but for example I don’t think ethos has any branch operations.
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by blep-arsh on Thursday January 08, 2026 @10:54PM
(#46549874)
There are specialized computation kernels compiled for NPUs. A high-level program (that uses ONNX or CoreML, for example) can decide whether to run the computation using CPU code, a GPU kernel, or an NPU kernel or maybe use multiple devices in parallel for different parts of the task, but the low-level code is compiled separately for each kind of hardware. So it's somewhat abstracted and automated by wrapper libraries but still up to the program ultimately.
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by indubioprorubik on Thursday January 08, 2026 @05:03PM
(#46547117)
YaWelco
metothe
clubpal
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by mjevans on Friday January 09, 2026 @12:29AM
(#46550385)
So it's a higher power DSP style device. Small transformers for flows. Sounds good for audio and maybe tailored video flow processing.
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by cookiengineer on Friday January 09, 2026 @01:23AM
(#46550687)
My question is: Isn't this exactly what SIMD has done before? Well, or SSE2 instructions?
To me, an NPU and how it's described just looks like a pretty shitty and useless FPGA that any alternative FPGA from Xilinx could easily replace.
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by recursivecaveat on Friday January 09, 2026 @01:39AM
(#46550757)
You definitely would use SIMD if you were doing this sort of thing on the CPU directly. The NPU is just a large dedicated construct for linear algebra. You wouldn't really want to deploy FPGAs to user devices for this purpose because that would mean paying the reconfigurability tax in terms of both power-draw and throughput.
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by imtringued on Friday January 09, 2026 @06:15AM
(#46552648)
Yes but your CPUs have energy inefficient things like caches and out of order execution that do not help with fixed workloads like matrix multiplication. AMD gives you 32 AI Engines in the space of 3 regular Ryzen cores with full cache, where each AI Engine is more powerful than a Ryzen core for matrix multiplication.
Re:Dell's CES 2026 chat was the most pleasingly un-AI briefi...
by cookiengineer on Saturday January 10, 2026 @04:21AM
(#46564155)
I thought SSE2 and everything that came after like AVX 512 or SSE4 were made for streaming, leveraging the cache only for direct access to speed things up?
Haven't used SSE instructions for anything other than fiddling around with it yet, so I don't know if I'm wrong in this assumption. I understand the lock state argument about cores due to always max 2 cores being able to access the same cache/memory... but doesn't this have to be identical for FPUs if we compare this with SIMD + AVX?
"You multiply matrices of INT8s."
"OH... MY... GOD"
NPUs really just accelerate low-precision matmuls. A lot of them are based on systolic arrays, which are like a configurable pipeline through which data is "pumped" rather than a general purpose CPU or GPU with random memory access. So they're a bit like the "synergistic" processors in the Cell, in the respect that they accelerate some operations really quickly, provided you feed them the right way with the CPU and even then they don't have the oomph that a good GPU will get you.