FPGAs are great because they are field-programmable and can adapt the hardware to a task. In all other aspects (density, power draw, cost) they are not that great.

We could consider something like Tilera chips working together with privileged "general compute" chips, the same way as current machines work with separate GPUs as privilege "vector compute" chips.