Ok... lemme take you down a rabbit hole.... imagine an FPGA, but with no routing hardware what so ever... just a vast sea of LUT (Look Up Tables) base cells with 4 bits of input and output, from each of its neighbors. A latch on the inputs is clocked like a chessboard... all of the white, then all of the black... this slows down processing, but completely removes timing issues.

Properly programmed, a large fraction of all the LUTs could be doing computation on every clock cycle. Imagine a 1024x1024 grid of these.... you could throw inputs in one side of the array, and get outputs every clock cycle, as everything is pipelined.

Programming.... no coherent ideas about that... I have some, involving working backwards from the output.

Lets assume 1024x1024, 100 Mhz clocking, and a FP16 taking 24x24 cells... you could have 42x42 computes on each and every clock cycle, about 175 Billion FP16 ops/second... for something that takes up the same space as 64 megabytes of static ram. Bear in mind, that's being very conservative without optimizations.

At present, I can simulate that 1024x1024 grid at about 37 Hz on my desktop.[1]

[1] https://github.com/mikewarot/Bitgrid