It is a totally different way of programming and thinking though and you need to spend a lot of thought on design to make it really fast. We could outpace a 3GHz CPU with a Zynq for image processing but it needs some really good FPGA people and the right split between CPU and FPGA. Nothing the average programmer wants to dive into, more something for cost and power sensitive markets with a relative fixed task, not needing ASICs yet. For networks it could pay off if you have the right amountof multipliers in the right places, but only then. As you wrote, don't leave the chip during the processing and you are good to go.