When you're judging whether Moore's law is dead, you need to separate out the improvements from changing the FPU layout and the improvements from lithography. And since the consumer chips don't dedicate a lot of space to neural net units, they're the best baseline.

It's cool that they made some neural net things go hugely faster. But they could make the same fundamental chip on 6-year-old foundries, only slower by a factor of 2 or 3. That really impressive number has nothing to do with Moore's law.