Re:What's new in Llama 2 and how to run it locally
by MediumOwl on Monday August 07, 2023 @07:39AM
(#37033104)
There's only mention of Nvidia GPUs on the web site, what about AMD?
[ Parent
| on HN ]
Re:What's new in Llama 2 and how to run it locally
by zzbzq on Monday August 07, 2023 @09:07AM
(#37033894)
Don't really work for AI. There might be a weird experimental driver for linux or something, I never got it to work.
Re:What's new in Llama 2 and how to run it locally
by MediumOwl on Monday August 07, 2023 @09:29AM
(#37034185)
Why not?
Re:What's new in Llama 2 and how to run it locally
by zzbzq on Monday August 07, 2023 @09:55AM
(#37034510)
They suck at it basically. My understanding is it's really the software, Nivida supports AI first-class with the drivers (or whatever.) AMD has next to nothing nothing.
Re:What's new in Llama 2 and how to run it locally
by lhl on Monday August 07, 2023 @11:27AM
(#37035777)
On Windows, llama.cpp has OpenCL support (CLBlast) and MLC LLM (https://mlc.ai/mlc-llm/docs/) has Vulkan acceleration.
On Linux, ExLlama and MLC LLM have native ROCm support, and there is a HIPified fork of llama.cpp as well.