Sure - not as fast or flexible, but relatively cheap and reprogrammable - good for experimentation, maybe for building a real-world proof-of-concept of a learning architecture ?

Funny you say that, if you look at the git repo for the blog there are a bunch of asic screenshots currently being committed.

This doesn't actually matter in practice, because either nobody produces your specific architecture in the first place, or the existing architectures are significantly more power inefficient.

FPGAs run at reduced clock speeds, but then again, you'd have to run the GPU or CPU at the same clock speed to obtain the same energy efficiency.