Thanks for clarifying.

The right kind of sea of LUTs can outperform anything even if it's clocked at 100 Mhz... the trick is to get a pipeline filled, instead of trying to outrun light.

Imagine an LLM with a new token every 10 nS

But you can't fit that "sea" into a processor core as a separate unit powered by unique instructions (like an FPU or ALU) because the speed demands the cache and ALU be there (or else it can't run at 3GHz any more). Once it's not that close it doesn't make sense to be on the same chip; maybe a neighboring chiplet. But anybody could do that.

It is a totally different way of programming and thinking though and you need to spend a lot of thought on design to make it really fast. We could outpace a 3GHz CPU with a Zynq for image processing but it needs some really good FPGA people and the right split between CPU and FPGA. Nothing the average programmer wants to dive into, more something for cost and power sensitive markets with a relative fixed task, not needing ASICs yet. For networks it could pay off if you have the right amountof multipliers in the right places, but only then. As you wrote, don't leave the chip during the processing and you are good to go.

> It's one of those things that seem like a good idea, but they just don't work out in practice.

GPGPU sucks a lot of air out of the room as well. There aren't many purely computational problems which FPGAs can solve better than a compute-optimized GPU; even though GPUs aren't quite as flexible, they clock a lot faster, they're cheaper, and they're easier to develop for.

I actually think there are a ton of problems which could benefit. But there is just very little mindshare. You basically have 1000x people working on building software to run efficiently on the GPU. No way FPGA touches that. It doesn't help that it's a specialized set of skills to build high performance circuit. Whereas running on a GPU is "just" software.

So even though I think FPGA could win out from an efficiency perspective there is just not enough people working on it.

That's been a historically really good bet for processor architectures -- developer volume beats hypothetical performance.

Essentially the only time it's gone the other way was when there was a toehold in a critical market, where the benefits were so obvious and profitable that they made up for the difficulty (e.g. graphics).

> FPGA LUTs are just way too slow

If, and of course that is a big if, you can repackage a (parallelizable) calculation into FPGA look-up tables and implement multiples of this (e.g. 8 to 80 times) then you can think maybe it's quicker than CPU at 3GHz.

However, you have to include DMA of the data to and fro. It's unlikely to be worth the very extensive effort of integrating two wildly different technologies.

On the other hand, it may not be a complicated calculation but FPGA can do much lower latency and smaller variance in latency (hello high-frequency traders). That is a very narrow niche.

A simple board with CPU and FPGA is the Arduino MKR Vidor 4000: ARM Cortex 32-bit CPU and Intel Cyclone 10 FPGA). Hardware cost: $85. Full suite of development software $1000 or more (although lesser tools are available for free.)

>However, you have to include DMA of the data to and fro. It's unlikely to be worth the very extensive effort of integrating two wildly different technologies.

That is exactly the part where having the FPGA next to the CPU helps... You can transparently access the CPU cache via an AXI slave port on the CPU on AMD's MPSoCs at a rate of up to 16 bytes per cycle and you get multiple of those.

Hmm, very interesting! Didn't know that wa possible.

> where doing something on a 3GHz CPU clock running multiple instruction parallel gets outperformed by LUTs that runs at 700MHz

Easy: go wide.

Make the FPGA-CPU interface four times wider on the FPGA side than the CPU side. Each tick of the CPU clock reads (or writes) one quarter of the bits.