News for hackers. Stuff that matters.

A comment on OpenLLM. Parent comment

Re:OpenLLM by aarnphm on Monday June 19, 2023 @05:06AM (#36388895)
Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
[ Parent | on HN ]
Re:OpenLLM by nacs on Monday June 19, 2023 @11:28AM (#36392192)
GPTQ support would be amazing (AutoGPTQ is an easy way to integrate GPTQ support - it's basically just importing autogptq and switching out 1 line in the model loading code).
[ Parent | on HN ]