News for hackers. Stuff that matters.
Re:Visualize Mixtral Moe by patrakov on Saturday December 23, 2023 @02:35AM (#38742390)
With a recent llama.cpp, Mixtral, when run using CPU only (8 threads), produces 4 tokens per second on a Dell Inspiron 7415 laptop with AMD Ryzen 7 5700U, upgraded to 64 GB of RAM. As this is an APU, even one layer is too much for the VRAM, so acceleration cannot be used.

There are no comments on this comment yet.