Energy-Efficient Llama 2 Inference on FPGAs via High Level Synthesis

Re:Energy-Efficient Llama 2 Inference on FPGAs via High Leve... by sandGorgon on Friday May 10, 2024 @02:46AM (#40316056)

if i want to play around with this at home on a fpga devkit...which fpga kit should i use ?

the Xilinx Virtex UltraScale+ VU9P FPGA prototyping boards seem to be 9000 USD. Anything in the 1000$ range ?

[ Parent | on HN ]

Re:Energy-Efficient Llama 2 Inference on FPGAs via High Leve... by almostgotcaught on Friday May 10, 2024 @02:56AM (#40316106)

ebay has a bunch of Alveo U30 cards for ~500. 500k luts, 3000 dsp slices, ~1M registers, even some DDR.

fair warning: one does not "play around" with an FPGA. they are the antithesis of user friendly.

[ Parent | on HN ]

Re:Energy-Efficient Llama 2 Inference on FPGAs via High Leve... by BonusPlay on Friday May 10, 2024 @03:00AM (#40316118)

Note, that paper provides environment for AWS FPGAs, which you can rent on per-hour basis.

As for cheaper FPGAs, the paper notes that the bottleneck is the size of on-chip memory. So I doubt it will be easy to find cheaper model to reduce costs.

Another hidden fee would be Vivado and Vitis (tooling) licenses, which you need for most upper-end FPGAs.

[ Parent | on HN ]

Re:Energy-Efficient Llama 2 Inference on FPGAs via High Leve... by sandGorgon on Friday May 24, 2024 @12:35AM (#40462821)

hi. this is very useful info thanks for this.

[ Parent | on HN ]