News for hackers. Stuff that matters.
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by chpatrick on Tuesday January 09, 2024 @04:55AM (#38924245)
Could you share what you use to run it on a single 3090? I'd love to try it!
Re:Mixtral 8x7B: A sparse Mixture of Experts language model by coder543 on Tuesday January 09, 2024 @08:46AM (#38926079)
ollama has been by far the easiest way for me, either on Linux directly (as I do now) or WSL2.