Because the NPU isn't for high-end inferencing. It's a relatively small coprocessor that is supposed to do bunch of tasks with high TOPS/watt without engaging the way more power hungry GPU.

At release time, the windows driver for example included few video processing offloads used by Windows Frameworks used for example by MS Teams for background removal - so that such tasks use less battery on laptops and free up CPU/GPU for other tasks on desktop.

For higher end processing you can use the same AIE-ML coprocessors various chips available previously from Xilinx and now under AMD brand.

> the same AIE-ML coprocessors

they're not the same - versal acaps (whatever you want to call them) have AIE1 arch while phoenix has AIE2 arch. there are significant differences between the two arches (local memory, bfloat16, etc.)

Phoenix has AIE-ML (what you call AIE2), Versal has choice of AIE (AIE1) and AIE-ML (AIE2) depending on chip you buy.

Essentially, AMD is making two tile designs optimized for slightly different computations and claims that they are going to offer both in Versal, but NPUs use exclusively the ML-optimized ones.