News for hackers. Stuff that matters.
Re:MiniGPT-4 by muyuu on Monday April 17, 2023 @09:10AM (#35600101)
quantized Vicuna runs ok-ish in my 16GB i7 laptop (onboard graphics) and the output is usable

see this comparison: https://old.reddit.com/r/LocalLLaMA/comments/12ezcly/compari...

these models quantised to 4bit should run in CPU set ups with 16GB of RAM + 16GB of swap (Linux) and perhaps other setups run similarly

There are no comments on this comment yet.