News for hackers. Stuff that matters.

A comment on Large language models are having their Stable Diffusion moment. Parent comment

Re:Large language models are having their Stable Diffusion m... by qumpis on Saturday March 11, 2023 @08:18PM (#35115165)
What's your ram usage for this?
[ Parent | on HN ]
Re:Large language models are having their Stable Diffusion m... by ddren on Saturday March 11, 2023 @08:39PM (#35115363)
The (quantized) 13B model is 7.6 GB on disk and the program uses around 8 GB to run. It runs without hitting the swap with just 9 GB assigned to WSL2.
[ Parent | on HN ]
Re:Large language models are having their Stable Diffusion m... by JoeMattie on Saturday March 11, 2023 @09:39PM (#35115850)
I've got the (4-bit quantized) 65B param model running at somewhat acceptable speed on an i9-7900. It uses around 55GB of RAM.
[ Parent | on HN ]