Re:Mistral 7B,The complete Guide of the Best 7B model
by tayo42 on Tuesday October 31, 2023 @02:45AM
(#38081055)
I wonder if it's possible to do something like folding at home but train an llm and have it always be open
[ Parent
| on HN ]
Re:Mistral 7B,The complete Guide of the Best 7B model
by brucethemoose2 on Tuesday October 31, 2023 @12:22PM
(#38087262)
Re:Mistral 7B,The complete Guide of the Best 7B model
by tayo42 on Tuesday October 31, 2023 @01:02PM
(#38087961)
wow didnt know that, thats really cool, ill have to play with it
Re:Mistral 7B,The complete Guide of the Best 7B model
by brucethemoose2 on Tuesday October 31, 2023 @02:09PM
(#38088970)
If you want to host for the AI Horde, Aphrodite has the most GPU throughput, and Koboldcpp is the easiest and quite flexible:
https://github.com/PygmalionAI/aphrodite-engine https://github.com/LostRuins/koboldcpp
Picking a model to host is quite a rabbit hole, but I am very interested in SciPhi and Zephyr at the moment.
I havent messed with Petals yet either, but swarm finetuning is very appealing... If thats even implemented atm.