News for hackers. Stuff that matters.

A comment on LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs. Parent comment

Re:LlamaF: An Efficient Llama2 Architecture Accelerator on E... by someguydave on Friday September 27, 2024 @11:24AM (#41671612)
gotta prototype the thing somewhere. If it turns out that the LLM algos become pretty mature I suspect accelerators of all kinds will be baked into silicon, especially for inference.
[ Parent | on HN ]
Re:LlamaF: An Efficient Llama2 Architecture Accelerator on E... by jsheard on Friday September 27, 2024 @11:32AM (#41671742)
That's the thing though, we're already there. Every new consumer ARM and x86 ASIC is shipping with some kind of NPU, the time for tentatively testing the waters with FPGAs was a few years ago before this stuff came to market.
[ Parent | on HN ]
Re:LlamaF: An Efficient Llama2 Architecture Accelerator on E... by PaulHoule on Friday September 27, 2024 @11:35AM (#41671787)
But the NPU might be poorly designed for your model or workload or just poorly designed.
[ Parent | on HN ]
Re:LlamaF: An Efficient Llama2 Architecture Accelerator on E... by mistrial9 on Friday September 27, 2024 @12:14PM (#41672371)
like this? https://www.d-matrix.ai/product/
[ Parent | on HN ]