News for hackers. Stuff that matters.
Re:Mistral AI Launches New 8x22B MOE Model by bevekspldnw on Thursday April 11, 2024 @11:14AM (#40003079)
There is a user called The Bloke on hugging face- they release pre quantized models pretty soon after the full size drop. Just watch their page and pray you can fit the 4 bit in your GPU.

I’m sure they are already working on it.

Re:Mistral AI Launches New 8x22B MOE Model by nathanasmith on Thursday April 11, 2024 @12:54PM (#40004287)
TheBloke stopped uploading in January. There are others that have stepped up though.
Re:Mistral AI Launches New 8x22B MOE Model by bevekspldnw on Thursday April 11, 2024 @02:20PM (#40005175)
Oh really? Who else should I be looking at?

That person is a hero, super bummed!

Re:Mistral AI Launches New 8x22B MOE Model by fzzzy on Thursday April 11, 2024 @03:38PM (#40005970)
TheBloke's grant ran out.
Re:Mistral AI Launches New 8x22B MOE Model by MPSimmons on Thursday April 11, 2024 @01:18PM (#40004541)
I think 4b for this is support to be over 70GB, so definitely still heavy hardware.
Re:Mistral AI Launches New 8x22B MOE Model by bevekspldnw on Thursday April 11, 2024 @02:40PM (#40005366)
Fucking hell, my A6000 is shy of that and I can’t reasonably justify picking up a second.