News for hackers. Stuff that matters.

A comment on MLC-LLM: GPT/Llama on consumer-class GPUs and phones.

Re:MLC-LLM: GPT/Llama on consumer-class GPUs and phones by TheObviousOne on Sunday April 30, 2023 @01:34PM (#35764740)
is there any integration with it to langchain?

+

Is there any optimization for LLM to run on RTX cards? 40XX,30XX I found out tha LLAMA.CPP is nice but I want to take advantage of my graphic cards also, and didn't found any documentations...

[ Parent | on HN ]
Re:MLC-LLM: GPT/Llama on consumer-class GPUs and phones by juliangoldsmith on Sunday April 30, 2023 @01:37PM (#35764768)
rllama has an OpenCL version, though I wasn't able to test it.
[ Parent | on HN ]