News for hackers. Stuff that matters.
Re:Mistral 7B,The complete Guide of the Best 7B model by tayo42 on Tuesday October 31, 2023 @02:45AM (#38081055)
I wonder if it's possible to do something like folding at home but train an llm and have it always be open
Re:Mistral 7B,The complete Guide of the Best 7B model by brucethemoose2 on Tuesday October 31, 2023 @12:22PM (#38087262)
Re:Mistral 7B,The complete Guide of the Best 7B model by tayo42 on Tuesday October 31, 2023 @01:02PM (#38087961)
wow didnt know that, thats really cool, ill have to play with it
Re:Mistral 7B,The complete Guide of the Best 7B model by brucethemoose2 on Tuesday October 31, 2023 @02:09PM (#38088970)
If you want to host for the AI Horde, Aphrodite has the most GPU throughput, and Koboldcpp is the easiest and quite flexible:

https://github.com/PygmalionAI/aphrodite-engine https://github.com/LostRuins/koboldcpp

Picking a model to host is quite a rabbit hole, but I am very interested in SciPhi and Zephyr at the moment.

I havent messed with Petals yet either, but swarm finetuning is very appealing... If thats even implemented atm.