News for hackers. Stuff that matters.
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by Casteil on Monday January 08, 2024 @11:59PM (#38922521)
Nice - that's still pretty solid.. although on a more typical 3060 or 3070 with less vram available, I probably wouldn't expect numbers quite that good.

My 14" M1 Max does around 30t/s on Mixtral Q4_K_M.

There are no comments on this comment yet.