LLama.cpp now has a web interface

Re:LLama.cpp now has a web interface by vitaminka on Wednesday July 05, 2023 @04:29PM (#36606662)

i’m curious, what’s is the approach for maintainable and decoupled various gpu backends?

[ Parent | on HN ]

Re:LLama.cpp now has a web interface by ggerganov on Wednesday July 05, 2023 @04:51PM (#36606916)

It was designed in #915 (read just the OP and the linked PRs at the end) and the implementation pretty much follows it closely, at least for the Metal backend. The CUDA and OpenCL backends are currently slightly coupled in ggml as they started developing before #915, but I think we'll resolve this eventually.

#915 - https://github.com/ggerganov/llama.cpp/discussions/915

[ Parent | on HN ]

Re:LLama.cpp now has a web interface by vitaminka on Wednesday July 05, 2023 @05:19PM (#36607293)

interesting decoupling method, ty :)

[ Parent | on HN ]