NPUs have so far been too narrow for even what they are intended for. Apple ships AMX/SME because their NPU has only a few data types supported and isn't flexible enough for all the things users actually want to do. AMD and Apple don't even try selling them as features for users to actually program for.
People bought graphic cards that were only good for one very specific workload, because that workload didn't fit the CPU architecture. While that eventually became another kind of general purpose architecture, it wasn't initially. Now we have NPUs and TPUs because the workload demands a specific shape of silicone.
At some point a model from two years ago that runs for a thousandth of the (dollar/energy) price of a frontier model becomes an extremely useful thing to have, if only as a subagent.