[Yup, "pipeline" here is the same term used to describe how modern CPUs get their performance. They, too, are pipelining their instruction execution so that many instructions can be in the process of executing at the same time. Though what I describe is a more extreme and specific kind of pipelining.]
This is kind of like having a bunch of multiplies in parallel. But it's useful for, for example, real-time calculations. You get to perform all the calculations you need every single cycle, no matter how complex; your results are just delayed by X cycles. Pipelines are usually also more efficient than straight parallelism (i.e. an 8 deep pipelined multiplier uses less silicon than 8 individual "serial" multiplier units).
Another interesting thing: In a previous life I built an FPGA based video processing device. It sat in an HDMI chain, so it had to be real-time. If that had been built with a GPU, most engineers would build the system to buffer up a frame, perform the processing, and then feed the processed frame out. That results in at least 1 frame of delay. In contrast, because we used an FPGA, it was simple to just pipeline the entire design and thus only needed to buffer up the few lines that we needed. This meant A) we needed no external memory (cheaper) and our latency was on the order of microseconds. In my travels with that job I ran into tons of other companies building video processing devices. They _all_ used frame buffers, which made their devices unacceptable for, e.g., gaming.
Just to be clear: CPUs will pipeline high-latency execution units. Most arithmetic operations, vector, integer, or floating point, will have a reciprocal throughput of 1 (i.e., you can issue a new operation every cycle) even if their latency will be 3 or 4 cycles. See Agner's optimization tables for actual counts. The major exception to that rule is division, which has a reciprocal throughput of ~5 cycles or so and a latency ~3-4 times that.
Just to clarify whilst I have your attention: isn't it a common practice to pipeline frame buffers too? For instance, Android has three frame buffers. IIRC, one is with the user space handing off display lists, one used for composition, and one is used by driver for rasterization?
In that case, how good, you reckon, would the perf be compared to FGPA?
No, not that latency. Latency from external input to external output.
Like the latency of receiving an ethernet frame to the point you can decide what to do with it, forward, modify, etc. Or reacting to sensor input.
CPUs process interrupts slowly: X86 CPUs running a non-realtime OS can take sweet 50 microseconds+ just to get to run interrupt service routine let alone doing something useful with the data. Oh, and generally no FPU, SIMD, etc. is available in this ISR [0]. So ISR generally queues the event (on Linux a Bottom Half and on Windows a DPC, deferred procedure call) to actually process the event. The queued events might execute tens of milliseconds later in bad cases.
Microcontrollers can sometimes process external interrupts pretty quickly, it's not impossible to be done in 100 nanoseconds. But it really depends, and you usually have to be pretty careful to achieve this. And even then you often have significant jitter in the processing time.
FPGAs can start to handle an external I/O event on the same clock cycle, and might very well be driving I/O output pin a clock cycle later, if no pipelining is required.
So an FPGA might be done in mere 10 nanoseconds in some ideal cases. With little to no jitter.
[0]: Well, you can do FPU/SIMD in interrupt service routine, but need to handle all register state storage on your own, and if you mess up, it'll cause very interesting bugs in the userland application software.
It's also worth noting that FPGAs are often much closer to the I/O than a CPU on a motherboard. Pin to pin latencies are much reduced in a situation where direct access to I/O is available. An Ethernet stack can sit on the same silicon without having to serialize data between the CPU and the north bridge.
If you're considering an FPGA, isn't the alternative likely bare-metal CPU programming? In that case, you have just as much control over thread scheduling as you do over FPGA timing don't you?
And then, with the SoCs that are everywhere now.. you are poking some memory mapped register, say you toggle a GPIO output value.. how long does it take to change on the actual physical pin? Is the GPIO peripheral part of the processor or some IP they bought in and then interconnected over an AXI bus? Is it buffered? It's all entirely impossible to say.
This by no means replaces an FPGA, but if you just want your pins to flip and code to run at a predictable interval, this gets you that. (As for reading pins, they have another peripheral on the die that will just timestamp when an event occurs. This is much more consistent than dealing with your OS's interrupt handler, but again, it's no FPGA.)
By far the simplest solution was to brute force it on a GPU. Probably took longer "single threaded", as there was no optimisation at all, but over O(10e6) pixels with a list of O(10e3) keypoints in shared memory it was basically instant.
It was a great lesson in premature optimisation. I could have spent days tweaking the CPU method with heuristics, sorting the inputs, etc. In the end it was less than 100 lines of OpenCL I think.
The real benefit of an FPGA is that you get to decide what happens in any given cycle. So rather than being able to multiply two numbers in a single cycle like on x86, you could make your FPGA design do, say, 20 multiplications in a single cycle (space allowing). Which means that you can now multiply 20 numbers in 1/10th of the time it would take on x86. (In reality I think you have something like four execution units capable of perfoming parallel ALU operations per cycle, depending on the family.)
So the latency benefit of an FPGA comes from flexible, almost (almost) unbounded potential for parallelism in a given cycle, not clock frequency. Hardware has to be designed to exploit this potential, otherwise it's not going to see any improvement in latency.
Anyways, just something that is maybe obvious once you're told it, but isn't always mentioned in discussions like this. It certainly something that I didn't fully appreciate before getting involved in hardware design.