News for hackers. Stuff that matters.
Re:Large language models are having their Stable Diffusion m... by koheripbal on Sunday March 12, 2023 @10:36AM (#35120513)
That's to be expected. 4 bits is too small. But the 8 bit 65b should outperform significantly.

Have you tried it?

Re:Large language models are having their Stable Diffusion m... by sebzim4500 on Sunday March 12, 2023 @01:54PM (#35123568)
Is 4 bits too small? Or is the quantization method just not sophisticated enough.

See https://arxiv.org/pdf/2210.17323.pdf

Re:Large language models are having their Stable Diffusion m... by enduser on Sunday March 12, 2023 @06:47PM (#35127428)
I’ll give it a try this evening.