News for hackers. Stuff that matters.

A comment on How to run Llama 13B with a 6GB graphics card. Parent comment

Re:How to run Llama 13B with a 6GB graphics card by jimmySixDOF on Monday May 15, 2023 @12:37AM (#35943926)
Some people are having some success speeding token rates and clawback on VRAM using a 0- group size flag but ymmv I did not test this yet (they were discussing gptq btw)
[ Parent | on HN ]

There are no comments on this comment yet.