Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Falcon 40B LLM (which beats Llama) now Apache 2.0. Parent comment
I still use GPTQ for 30B, but even CPU generates quickly enough at q5_1 on modern hardware.
There are no comments on this comment yet.
I still use GPTQ for 30B, but even CPU generates quickly enough at q5_1 on modern hardware.