News for hackers. Stuff that matters.

A comment on Large language models are having their Stable Diffusion moment. Parent comment

Re:Large language models are having their Stable Diffusion m... by koheripbal on Sunday March 12, 2023 @11:35AM (#35121304)
With the 4bit quantization they needed to make it is like a jpeg with only 16 pixels per "thought" parameter.

It's going to perform like a brilliant person with brain injury. 8 bit is far better and it would be interesting to see if they can get it working on any reasonable hardware.

[ Parent | on HN ]
Re:Large language models are having their Stable Diffusion m... by sebzim4500 on Sunday March 12, 2023 @01:38PM (#35123296)
Apparently if you quantize carefully you can avoid any loss of performance.

See https://arxiv.org/pdf/2210.17323.pdf

[ Parent | on HN ]
Re:Large language models are having their Stable Diffusion m... by BulgarianIdiot on Sunday March 12, 2023 @03:46PM (#35125241)
The first proofs of concept are rarely ideal. But you know no one will stop here. On the software side, on the hardware side.
[ Parent | on HN ]