News for hackers. Stuff that matters.

varunvummadi

Re:Mistral AI Launches New 8x22B MOE Model on Tuesday April 09, 2024 @10:44PM (#39986471)
The easiest is to use vllm (https://github.com/vllm-project/vllm) to run it on a Couple of A100's, and you can benchmark this using this library (https://github.com/EleutherAI/lm-evaluation-harness)
Re:Mistral AI Launches New 8x22B MOE Model on Tuesday April 09, 2024 @10:02PM (#39986252)
It beats the old GPT4 version in lmsys benchmark you can check it out here https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... but Command R is commercially licensed We can assume that mistral will do a better job.
Re:Mistral AI Launches New 8x22B MOE Model on Tuesday April 09, 2024 @09:33PM (#39986116)
Not sure trying to download the torrent and checking it out
Re:Mistral AI Launches New 8x22B MOE Model on Tuesday April 09, 2024 @09:33PM (#39986111)
They Just announced their new model on Twitter, which you can download using torrent
Re:Funny Take on OpenAI on Wednesday April 03, 2024 @09:07PM (#39925393)
Should appreciate the dedication of the person who made this website, haha
Re:Groq runs Mixtral 8x7B-32k with 500 T/s on Monday February 19, 2024 @02:42PM (#39433818)
So please let me know if I am wrong are you guys running a batch size of 1 in 500 GPU's? then why are the responses almost instant if you guys are using batch size 1 and also when can we expect bring your own fine tuned models kind of thing. Thanks!
That is true cursor extension is awesome