News for hackers. Stuff that matters.
Re:What's new in Llama 2 and how to run it locally by lhl on Monday August 07, 2023 @11:27AM (#37035777)
On Windows, llama.cpp has OpenCL support (CLBlast) and MLC LLM (https://mlc.ai/mlc-llm/docs/) has Vulkan acceleration.

On Linux, ExLlama and MLC LLM have native ROCm support, and there is a HIPified fork of llama.cpp as well.

There are no comments on this comment yet.