News for hackers. Stuff that matters.

A comment on GPU Embedding with GGML. Parent comment

Re:GPU Embedding with GGML by vanillax on Monday October 16, 2023 @12:23PM (#37902160)
- GPTQ: pure gpu inference, used with AutoGPTQ, exllama, exllamav2, offers only 4 bit quantization

what is autoGTPTQ and exllama, what do it mean it only works with AutoGPTQ and exllama? Are those like TensorFlow Frameworks?

[ Parent | on HN ]

There are no comments on this comment yet.