News for hackers. Stuff that matters.
Re:GPU Embedding with GGML by 3abiton on Monday October 16, 2023 @07:59AM (#37898553)
Isn't ggml supposed to be running on CPU? It seems that this can run over any GPU, not necessarily just NVIDIA
Re:GPU Embedding with GGML by wizzard0 on Monday October 16, 2023 @08:18AM (#37898776)
CPU is still the first-class option, but GGML also supports Metal, cuBLAS and clBLAST for hw acceleration
Re:GPU Embedding with GGML by louiskw on Monday October 16, 2023 @10:49AM (#37900605)
Would be great to support more GPU, Metal is the only graphics we support atm with CPU as a fallback
Re:GPU Embedding with GGML by wizzard0 on Tuesday October 17, 2023 @12:22AM (#37910584)
cuBLAS is cuda, clBLAST is OpenCL