News for hackers. Stuff that matters.
Re:GPU Embedding with GGML by smcleod on Monday October 16, 2023 @08:35AM (#37898979)
GGML has been replaced with GGUF now and GGML is no longer getting any updates.

GPU offloading for GGUF/GGML has been available for quite a long time in Text Generation WebUI and works very well, but isn’t nearly as fast as GPTQ or the new AWQ format.

Re:GPU Embedding with GGML by leetharris on Monday October 16, 2023 @08:43AM (#37899051)
GGUF is the file format, but GGML is still the framework