Re:Large language models are having their Stable Diffusion m...
by enduser on Saturday March 11, 2023 @10:25PM
(#35116216)
I’m getting better results with 30B unquantized (f16) than with 65B at 4 bits, FWIW. On a Mac Studio with 128GB RAM.
[ Parent
| on HN ]
Re:Large language models are having their Stable Diffusion m...
by wjessup on Saturday March 11, 2023 @10:33PM
(#35116276)
post example prompts and results please?
Re:Large language models are having their Stable Diffusion m...
by koheripbal on Sunday March 12, 2023 @10:36AM
(#35120513)
That's to be expected. 4 bits is too small. But the 8 bit 65b should outperform significantly.
Have you tried it?
Re:Large language models are having their Stable Diffusion m...
by sebzim4500 on Sunday March 12, 2023 @01:54PM
(#35123568)
Is 4 bits too small? Or is the quantization method just not sophisticated enough.
Re:Large language models are having their Stable Diffusion m...
by enduser on Sunday March 12, 2023 @06:47PM
(#35127428)
I’ll give it a try this evening.
Re:Large language models are having their Stable Diffusion m...
by sebzim4500 on Sunday March 12, 2023 @01:53PM
(#35123559)
From what I can tell, the quantization being used there is extremely naive.
If they were using techniques like GPTQ, presumably it would work much better.