LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs

Re:LlamaF: An Efficient Llama2 Architecture Accelerator on E... by bee_rider on Friday September 27, 2024 @02:24PM (#41673974)

LLM inference is a small task built into some other program you are running, right? Like an office suite with some sentence suggestion feature, probably a good use for an LLM, would be… mostly office suite, with a little LLM inference sprinkled in.

So, the “ASIC” here is probably the CPU with, like, slightly better vector extensions. AVX1024-FP16 or something, haha.

[ Parent | on HN ]

Re:LlamaF: An Efficient Llama2 Architecture Accelerator on E... by p1esk on Friday September 27, 2024 @09:08PM (#41677058)

would be… mostly office suite, with a little LLM inference sprinkled in.

No, it would be LLM inference with a little bit of an office suite sprinkled in.

[ Parent | on HN ]