Re:Visualize Mixtral Moe
by patrakov on Saturday December 23, 2023 @02:35AM
(#38742390)
With a recent llama.cpp, Mixtral, when run using CPU only (8 threads), produces 4 tokens per second on a Dell Inspiron 7415 laptop with AMD Ryzen 7 5700U, upgraded to 64 GB of RAM. As this is an APU, even one layer is too much for the VRAM, so acceleration cannot be used.
[ Parent
| on HN ]
There are no comments on this comment yet.