News for hackers. Stuff that matters.
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by 3abiton on Tuesday January 09, 2024 @03:05AM (#38923579)
Have you tried the 2x 3090 setup? Using nvlink or SLI?
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by coder543 on Tuesday January 09, 2024 @08:44AM (#38926068)
I have not personally gotten to test things that way
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by Tepix on Tuesday January 09, 2024 @01:56PM (#38930308)
I tried it, with NVlink. The speedup during inference is negligible. You'll probably benefit more during training.
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by kwerk on Tuesday January 09, 2024 @02:21PM (#38930595)
I use 2x 3090, no nvlink though. I’d read it doesn’t help that much but not well read on any improvement.