And I’m not even touching the fact that FPGAs (which I love to work with!) are a pain to program and optimize for.

It’s not that you can’t use them, and there may be narrow cases where they have a benefit. There are always trade-offs to be made.

But that doesn’t make them the best hardware for AI (there’s too much overhead) and it doesn’t change the fact that they’re much harder and slower than bare metal CUDA to optimize for, in a fast moving field.

Look at it this way: if you have a fixed DL architecture and a fixed set of network weights, there’s no question that you could come up with an ASIC that only does that, that will be cheaper than an FPGA or GPU, faster, and use lower power too. It will be the perfect inference solution for that specific problem.

It would also be obsolete in a few months, before the chip comes back from the fab.

That means an ASIC.