News for hackers. Stuff that matters.

A comment on Large language models are having their Stable Diffusion moment. Parent comment

Re:Large language models are having their Stable Diffusion m... by MacsHeadroom on Saturday March 11, 2023 @07:54PM (#35114952)
13B in 4bit runs on your average 8GB VRAM consumer video card at fast enough speeds.

The GP's info is outdated. Apparently they don't realize how quickly LLaMA is being developed on. It's just like the early days of SD.

[ Parent | on HN ]
Re:Large language models are having their Stable Diffusion m... by zamadatix on Saturday March 11, 2023 @08:27PM (#35115255)
llama.cpp runs on the CPU anyways, the whole conversation is on a false premise. The only hard thing about running it on consumer hardware is the quantization step (where the largest RAM requirements come in) and that's not even a technical problem since you could just redistribute the quantized version if it weren't for the license of the model.
[ Parent | on HN ]
Re:Large language models are having their Stable Diffusion m... by smy20011 on Saturday March 11, 2023 @08:16PM (#35115150)
Maybe out-dated by 1 day LOL.
[ Parent | on HN ]