Well kinda. The flexibility comes with a cost. Clock speeds are lower and for the same logic it's going to be less power efficient than a GPU or CPU or ASIC. Logic element density will also be lower so it'd be impossible to reproduce the entirety of a GPU or modern CPU (huge caches etc) on a similar-sized FPGA. I'd also imagine that the physical distance of gate paths are highly optimized in a CPU such that more gates can be traversed per clock tick there than in an FPGA.

If you have very specific compute pipelines with "colocated" (there's probably a better word -- basically data directly on the wire, not in memory lookups) data in parallel an FPGA can offer very high compute throughput.

But if you're having to do memory lookups and such for your algorithms then the memory bus is going to be your bottleneck and most of the FPGA's advantages are out the window. At that point sure you've got a few places where the FPGA does a few operations at once, but ultimately you just have an early 1990's speed processing pipeline with a few hardware-based micro-optimizations. A GPU and its custom designed baked in logic around memory buses and caches and such are going to blow it out of the water in terms of logical efficiency, power draw, heat dissipation, logic element count, and clock speed. Total throughput will be several orders of magnitude greater.

So, it depends what you're trying to do. It's dangerous to say FPGA is always badass. Sometimes they are, and sometimes they're not even close.