FPGAs run at reduced clock speeds, but then again, you'd have to run the GPU or CPU at the same clock speed to obtain the same energy efficiency.
Do you mean as a specialized processor for high speed inference, or as if the neural net including weights was directly synthesized into the FPGA architecture?
Neither use case would be as powerful as other options. FPGAs don’t provide infinite leeway for implementing completely custom logic. They have a finite number of blocks that can be reconfigured in certain ways as synthesized by the software. This architecture isn’t a good fit for the large LLMs we think about which need a lot of high bandwidth memory access. You can connect an FPGA to high speed memory, but the current crop of GPUs are going to be much better at the job.
What's exciting about FPGAs is that they make "custom" chips affordable for colleges and even individuals... At this point in time, with the AI world fixated on LLMs, it seems that there's an opportunity for small groups/individuals to use FPGAs to experiment with more animal-like ground-up approaches to intelligence.