Re:GPU Embedding with GGML
by 3abiton on Monday October 16, 2023 @07:59AM
(#37898553)
Isn't ggml supposed to be running on CPU? It seems that this can run over any GPU, not necessarily just NVIDIA
[ Parent
| on HN ]
CPU is still the first-class option, but GGML also supports Metal, cuBLAS and clBLAST for hw acceleration
Would be great to support more GPU, Metal is the only graphics we support atm with CPU as a fallback