News for hackers. Stuff that matters.
Re:Jamba: Production-grade Mamba-based AI model by patrakov on Thursday March 28, 2024 @05:20PM (#39857449)
Mixtral 8x7b runs well (i.e., produces the correct output faster than I can read it) on a modern AMD or Intel laptop without any use of a GPU - provided that you have enough RAM and CPU cores. 32 GB of RAM and 16 hyperthreads are enough with 4-bit quantization if you don't ask too much in terms of context.

P.S. Dell Inspiron 7415 upgraded to 64 GB of RAM here.

There are no comments on this comment yet.