Re:Mixtral 8x7B: A sparse Mixture of Experts language model
by patrakov on Tuesday January 09, 2024 @02:26AM
(#38923382)
Just as another data point, a CPU-only setup with Q5_K_M would give you roughly 4 tokens per second on a Ryzen laptop (Dell Inspiron 7415 upgraded to 64 GB of RAM).
[ Parent
| on HN ]
There are no comments on this comment yet.